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ARI  Research  Reports  and  Technical  Reports  are  intended  for  sponsors  of 
R&D  tasks  and  for  other  research  and  military  agencies.  Any  findings  ready 
for  implementation  at  the  time  of  publication  are  presented  in  the  last  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recom¬ 
mendations  for  official  action  normally  are  conveyed  to  appropriate  military 
agencies  by  briefing  or  Disposition  Form. 


FOREWORD 


The  Presidio  of  Monterey  Field  Unit  of  the  U.S.  Army  Research  Institute 
has  as  its  primary  mission  the  execution  of  research  to  improve  training  to 
better  meet  unit  mission  requirements.  One  aspect  of  such  training  has  been 
the  area  of  foreign  language. 

The  Presidential  Commission  Report  (1979)  and  the  1982  Government  Account¬ 
ing  Office  (GAO)  report  entitled  "Weaknesses  in  the  Resident  Language  Training 
System  of  Defense  Language  Institute  Affect  the  Quality  of  Trained  Linguists" 
emphasized  the  need  to  identify  alternative  methodologies  for  teaching  languages. 

The  Defense  Language  Institute  Foreign  Language  Center  (DLIFLC)  currently 
uses  the  Progressive  Skill  Integration  (PSI)  approach,  a  functional  skill¬ 
building  approach  to  language  learning  that  progresses  through  a  number  of 
stages,  beginning  with  the  perception  of  new  concepts  and  culminating  with  the 
acquisition  of  working  communication  skills.  Though  this  system  has  been 
effective,  the  DLIFLC  and  other  Army  language  trainers  continue  to  examine 
methods  to  improve  training. 

This  report  provides  a  comprehensive  summary  of  a  research  effort  that 
compared  and  evaluated  three  of  these  methods:  the  suggestopedia,  a  flexible- 
scheduling  methodology,  and  the  standard  DLIFLC  methodology  currently  used. 
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A  COMPARISON  OF  INNOVATIVE  TRAINING  TECHNIQUES  AT  THE  DEFENSE  LANGUAGE  INSTITUTE 
FOREIGN  LANGUAGE  CENTER 


EXECUTIVE  SUMMARY 


Requirement: 

The  Defense  Language  Institute  Foreign  Language  Center  (DLIFLC)  and  the 
Soldier  Support  Center  (SSC)  requested  that  the  Army  Research  Institute  (ARI) 
assess  the  effectiveness  of  and  students'  attitudes  toward  suggestopedia 
instruction  relative  to  the  standard  DLIFLC  instruction  currently  employed. 
The  study  also  included  a  comparison  of  a  flexible-scheduling  methodology 
identified  by  the  DLIFLC  as  a  target  of  opportunity. 


Procedure: 

Fifty  junior  enlisted  Army  and  Navy  personnel  were  initially  examined. 
Twenty  Army  students  received  the  suggestopedia  instruction,  and  20  received 
the  standard  DLIFLC  instruction.  Ten  Army  and  Navy  students  received  the 
flexible-scheduling  instructional  methodology. 

Achievement  tests,  a  Proficiency  Advancement  Test  (PAT),  and  face-to-face 
oral  interviews  were  used  as  criteria  of  training  effectivness.  Achievement 
tests  were  administered  after  the  presentation  of  a  block  of  instruction  called 
a  module.  The  PAT  and  oral  interviews  were  administered  to  each  group  when 
they  completed  the  course  materials  comprising  Term  I,  the  period  used  for  this 
study.  Completion  dates  were  10  weeks  for  the  suggestopedia  methodology,  14 
weeks  as  retrospectively  observed  for  the  flexible-scheduling  methodology,  and 
15  weeks  as  normally  planned  for  the  standard  DLIFLC  methodology.  Attitudes 
were  assessed  at  the  beginning  and  end  of  the  term,  and  at  weekly  intervals. 
Student  demographic  variables  were  represented  by  military  rank,  military 
occupational  specialty  (MOS),  age,  years  of  military  service,  educational 
level,  prior  language  training,  and  gender.  Additional  descriptive  variables 
for  the  student  population  included  Defense  Language  Aptitude  Battery  (DLAB) 
scores,  General  Technical  (GT)  scores,  and  a  pretest  of  attitudes  about  the 
potential  advantages  of  learning  Russian  and  about  foreign  languages  in  general. 


Findings: 

The  available  data  showed  that  the  suggestopedia  methodology  was  not  as 
effective  after  10  weeks  of  study  as  the  flexibly  scheduled  instruction  after 
14  weeks  or  the  standard  DLIFLC  instruction  after  15  weeks.  In  fact,  the 
suggestopedia  group  had  significantly  lower  scores  on  all  measures  of  academic 
performance  with  the  exception  of  results  on  the  listening  component  of  the  PAT 
and  the  oral  interviews.  Measures  of  attitudes  indicated  that  the  suggestopedia 
group  was  more  positive  toward  their  instructor (s )  during  the  first  4  weeks, 


EXECUTIVE  SUMMARY  (Continued) 


but  then  fell  to  the  level  of  the  two  DLIFLC  groups.  On  attitudes  compared  at 
the  conclusion  of  the  course,  the  suggestopedia  group  generally  felt  more 
comfortable  about  its  ability  to  use  the  Russian  language  and  felt  that  less 
effort  had  been  required  when  compared  to  the  other  groups.  The  flexibly 
scheduled  and  standard  DLIFLC  groups  felt  more  positive  about  their  course 
materials,  but  also  had  more  hands-on  (workbooks)  and  take-home  materials 
(workbooks  and  testbooks).  A  comparison  between  pre-  and  posttest  results 
indicated  no  attitude  changes  within  these  two  groups. 

Utilization  of  Findings: 

This  research  implies  that  suggestopedia,  as  a  whole,  would  not  justify  a 
change  in  the  standard  DLIFLC  instructional  methodology.  However,  suggestopedia' s 
use  as  an  enrichment  adjunct  to  the  curriculum  may  warrant  further  research. 
Additionally,  components  of  the  suggestopedia  methodology  may  affect  factors 
such  as  student  attrition  and  memory  retention,  thereby  warranting  further 
research. 

The  evaluation  of  the  flexible-scheduling  methodology  indicates  that 
students  can  progress  through  the  curriculum  at  a  faster  pace  than  usual. 

Further  research  may  be  necessary  to  determine  whether  this  is  an  effect  of  the 
instruction  or  the  curriculum. 
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A  COMPARISON  OF  INNOVATIVE  TRAINING  TECHNIQUES 
AT  THE  DEFENSE  LANGUAGE  INSTITUTE  FOREIGN  LANGUAGE  CENTER 


INTRODUCTION 


As  part  of  the  military’s  continued  efforts  to  improve  the  quality  of  for¬ 
eign  language  training,  the  Soldier  Support  Center  (SSC)  and  Defense  Language 
Institute  Foreign  Language  Center  (DLIFLC)  submitted  a  Concept  Evaluation  Plan 
(CEP)  to  the  Training  and  Doctrine  Command  (TRADOC)  for  the  evaluation  of  an 
innovative  training  methodology  called  suggestopedia.  In  September,  1984, 

TRADOC  approved  the  CEP  and  a  contract  was  awarded  to  the  Lozanov  Learning  Sys¬ 
tems,  Inc.  (LLSI)  to  provide  a  resident  course  of  instruction  using  the  suggest¬ 
opedia  methodology.  The  Army  Research  Institute  (ARI)  was  asked  to  conduct  the 
evaluation  of  suggestopedia  as  compared  to  the  standard  instructional  methodology 
currently  used  at  the  DLIFLC.  A  third  methodology,  using  flexible  scheduling 
of  the  presentation  of  materials,  was  modified  from  the  standard  DLIFLC  course 
of  instruction,  identified  as  a  "target  of  opportunity,"  and  added  to  the  study 
for  evaluation.  The  objective  of  the  research  was  to  compare  the  effectiveness 
of  the  three  methodologies  using  measures  of  academic  performance,  and  analysis 
of  students’  attitudes  about  the  respective  instructional  methodology. 

The  suggestopedia  methodology  is  a  unified  system  of  instruction  character¬ 
ized  by  a  variety  of  techniques  emphasizing  a  relaxed  and  positive  learning 
atmosphere.  The  instruction  is  delivered  in  situational  contexts  maximizing 
the  use  of  the  oral  communicati  c  skills  (proficiency).  The  standard  DLIFLC 
methodology  used  a  Progressive  Skill  Integration  (PSI)  approach,  which  is  a 
functional  approach  to  language  teaching  that  stresses  the  integration  of  the 
various  components  of  language  (i.e.,  pronunciation,  grammar,  vocabulary,  writ¬ 
ing  systems,  etc.)  into  communication  skills.  It  is  a  progressive  approach  in 
that  students  advance  through  a  number  of  stages  beginning  with  the  perception 
of  new  concepts  and  culminating  with  the  acquisition  of  working  communication 
skills.  The  flexible-scheduling  treatment  is  similar  to  the  standard  DLIFLC 
methodology  except  that  the  former  uses  a  pacing  of  the  presentation  of  mate¬ 
rials  based  upon  group  readiness  rather  than  a  fixed  schedule  for  the  presen¬ 
tation  of  materials.  Further  discussion  characterizing  the  methodologies  may 
be  found  later  in  this  chapter  under  the  subtitle,  "Description  of  Instructional 
Treatments." 

Data  from  this  study  may  be  used  in  formulating  subsequent  research  efforts 
on  components  of  effective  instruction  that  characterize  the  methodologies  (e.g., 
functional  practice,  positive  reinforcement,  cueing,  positive  role  modeling). 
Individual  components  found  effective  may  be  incorporated  throughout  a  course 
of  instruction. 


Background 

A  review  of  the  literature  indicates  mixed  findings  regarding  suggestive- 
accelerated  learning  or  suggestopedia.  In  addition,  there  is  little  evidence 
reflecting  the  study  of  suggestopedia  in  a  military  setting  or  with  military 
personnel. 


1 


One  review  conducted  within  the  Federal  government  was  a  self  report 
experience  with  National  Security  Agency  personnel  at  a  five-week 
suggestopedia  Russian  Course  cited  by  Shitama  (1982)*  Overall  remarks  made  by 
the  ten  NSA  students  and  two  NSA  instructors  (who  only  attended  the  one-week 
Lozanov  instructor  training  workshop)  concluded  that  while  some  features  of 
suggestopedia  were  interesting  and  helpful,  such  as  the  use  of  music  and 
relaxation  techniques,  the  collective  methodology  is  not  recommended  for 
individuals  requiring  a  thorough  grasp  of  the  language.  A  report  which 
involved  Special  Forces  military  personnel  learning  German  with  the 
suggestopedia  methodology  was  conducted  at  Fort  Devens,  Massachusetts. 

Findings  indicated  a  reduction  in  course  length  from  12  to  6  weeks.  However, 
iO  of  the  12  students  had  been  previously  exposed  to  German  either  in  school 
or  overseas  (Dhority,  1984). 

Studies  conducted  by  Lozanov,  the  creator  and  developer  of  suggestopedia, 
suggest  high  success  rates  covering  a  range  of  topics  from  foreign  language 
training  to  training  in  mathematics  at  virtually  all  age  levels  (Lozanov, 
1978).  One  example  is  the  teaching  of  a  year  of  traditional  foreign  language 
curriculum  in  three  and  one-half  months  (Lozanov,  1975).  However,  these 
findings  and  others  indicating  high  success  rates  have  been  found  to  lack 
"scientific  validity"  as  noted  by  Bancroft  (1976)  and  Scovel  (1979)  in  his 
review  of  Lozanov's  suggestopedia. 

Kline  (1974)  found  in  many  of  Lozanov's  studies  highly  motivated  adults, 
experienced  language  teachers,  and  perfect  attendance  records  which  may  result 
in  high  success  rates  regardless  of  the  instructional  methodology  used.  In  an 
attempt  to  replicate  some  of  Lozanov's  work,  Kline  indicated  that  vocabulary 
achievement  was  approximately  half  of  that  reported  by  Lozanov. 

Other  studies  (Benitez-Bordon  &  Schuster,  1976)  indicate  positive  results 
but  only  for  two-hour  sessions  once  a  week  and  with  no  control  group.  An 
experiment  in  teaching  beginning  Russian  (Kurkov,  1971)  indicated  higher 
success  rates  for  suggestopedia  over  traditional  instruction,  but  it  was  noted 
that  approximately  twice  as  many  students  in  the  suggestopedia  group  had  prior 
formal  Russian  language  training.  Additionally,  the  students  in  the 
suggestopedia  group  indicated  that  they  needed  to  spend  more  time  than  usual 
in  outside  preparation  such  as  vocabulary  copying  and  reading  grammar.  Renard 
(1976)  reviewed  a  study  using  suggestopedia  to  learn  French  but  found  that  the 
suggestopedia  group  was  comprised  of  volunteers  while  the  control  group  was 
not. 


Wagner  and  Tilney  (1982)  conducted  an  experiment  comparing  suggestopedia 
with  traditional  instructional  methods  for  learning  over  five  weeks.  Results 
indicated  no  significant  improvement  by  the  suggestopedia  group.  In  fact,  it 
was  found  that  the  traditional  group  learned  significantly  more  vocabulary. 

Other  studies  evaluated  characteristics  found  in  the  suggestopedia 
methodology.  Several  findings  indicated  an  increase  in  performance  levels  for 
verbal  learning  and  retention  when  students  were  provided  with  some  relaxation 
techniques  to  reduce  stress  before  testing  on  difficult  material  (Straughn  & 
Dufort,  1969;  Chaney  &  Andreasen,  1972).  However,  studies  by  Martin  and 
Schuster  (1977)  and  Lipsitt  (1963)  found  that  there  seemed  to  be  an  optimal 
level  of  stress  for  learning.  Some  students  were  found  to  learn  better  if 
they  felt  a  certain  amount  of  stress.  Overall  the  findings  seemed  to  indicate 


2 


I 

I 


1 


[ 


a  curvilinear  relationship  between  stress  and  learning.  Learning  increased  as 
stress  increased  up  to  an  optimum  level,  after  which  learning  fell  off  as  a 
function  of  increases  in  stress.  This  demonstrates  the  Yerkes-Dodson  Law. 

For  further  reading  see  Sanders j  Eng,  and  Murph  (1985). 

The  components  of  music  and  imagery  adapted  from  suggestopedia  were 
studied  for  their  separate  and  combined  impact.  Stein  (1982)  found  that  the 
addition  of  music  and  imagery  together  and  music  separately  indicated  a 
significant  increase  in  the  retention  of  vocabulary  for  college  students  when 
compared  to  the  control  group  without  treatment.  Subjects  in  the  music  plus 
imagery  group  heard  Handel's  Water  Music,  and  the  experimenter  read  aloud  the 
words  to  be  studied.  Subjects  in  the  music  only  group  heard  just  the  music. 
The  control  group  subjects  had  neither  the  words  spoken  nor  the  music,  they 
had  only  the  words  to  study. 

Since  this  study  deals  with  a  comparison  of  specific  instructional 
strategies  and  not  learning  theory  in  general,  the  following  references  are 
provided  for  futher  reading  as  an  overview  of  instructional  strategies.  Gagne 
and  Dick  (1982)  review  educational  research  and  its  implications  for  effective 
instruction.  Bush  (unpublished  manuscript)  discusses  effective  instructional 
strategies  according  to  subject  matter  and  student  demographics.  Weinstein 
and  Mayer  (1985)  describe  the  application  of  learning  theory  in  the  classroom. 

These  citations  seem  to  indicate  a  need  for  more  empirical  data  on  the 
suggestopedia  methodology  as  well  as  a  need  to  evaluate  the  specific 
components  of  suggestopedia  instruction  which  may  enhance  learning. 

Description  of  Instructional  Methodologies 

The  treatments  are  first  described  and  then  compared  as  actually  applied 
in  the  study.  Certain  modifications  were  necessitated  by  the  limitations 
imposed  by  the  experimental  conditions.  Modifications  to  the  treatments, 
particularly  suggestopedia,  were  done  with  the  collaboration  and  approval  of 
the  contractor  for  suggestopedia,  Lozanov  Learning  Systems,  Inc.,  and  the 
DLIFLC. 

Suggestopedia  Instructional  Methodology.  The  suggestopedia  method  is 
designed  to  approximate  many  conditions  of  the  early  childhood  learning 
process.  It  stresses  role-playing,  student  participant  dialogue,  and  a 
totally  positive  approach.  The  instructor(s)  encourage  interaction  of  the 
entire  class  through  the  use  of  positive  reinforcement,  stress  relaxation,  and 
confidence  building  techniques  demonstrated  or  modeled  by  the  instructor.  The 
classroom  environment  is  also  relaxed,  comfortable,  and  non-threatening. 
Subliminal  stimuli  are  also  important:  the  room,  furniture,  lighting,  and 
positioning  of  class  members  are  carefully  selected  and  arranged  to  enhance 
student  comfort  while  facilitating  student-teacher  interaction. 

A  phase  of  directed  passivity  alternates  with  an  active  phase.  During  the 
passive  phase  the  student  listens  to  material  orally  presented  by  the 
instructor  with  music  in  the  background  and  without  specific  directions  for 
retaining  the  material.  During  the  active  phase  students  role  play  and  enact 
situations  based  on  the  thematic  suggestions  from  the  instructor(s) . 
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Suggestopedia  also  emphasizes  the  importance  of  the  instructor  as  a  figure 
of  authority  and  prestige  while  providing  a  relaxed,  comfortabLe,  and 
nonthreatening  classroom  environment. 

The  suggestopedia  method  of  instruction  claims  to  ensure  the  acquisition 
of  communication  skills  with  emphasis  on  verbal  skills  but  also  including 
reading  and  writing  skills  (Sterling,  1984). 

Standard  DL1FLC  Instructional  Methodology.  The  methodology  described  for 
the  standard  DLIFLC  instruction  is  derived  from  a  Progressive  Skill 
Integration  (PSI)  process  which  involves  the  functions,  contents,  and  accuracy 
components  of  the  Interagency  Language  Roundtable  Skill  Level  Descriptions 
(see  appendix  F  for  a  detailed  listing).  (Since  a  measure  of  language 
accuracy  is  uncertain,  especially  at  the  beginning  of  the  course,  language 
accuracy  is  always  measured  in  relation  to  the  skill  level  descriptions  and 
not  to  perfect  grammar  models  [DLIFLC  Pamphlet  350-10,  1982]). 

The  PSI  process  uses  a  functional  approach  to  language  instruction  that 
stresses  the  integration  of  various  components  of  language  such  as 
pronunciation,  grammar,  vocabulary,  and  writing  into  the  communication  skills 
of  proficiency.  The  skills  of  listening,  speaking,  reading,  and  writing  are 
first  developed  through  a  series  of  learning  activities  based  upon  skill 
acquisition  or  achievement.  Exercises  such  as  memorizing  dialogues,  questions 
and  answers,  paraphrasing,  role-playing,  dictation,  and  translation  are 
carried  out  through  a  process  moving  from  conceptualization,  through 
familiarization,  and  variation.  Then  a  further  step  to  stimulation  and 
communication  brings  the  student  to  application  or  proficiency  activities. 
Teaching  strategies  such  as  audio-lingual  (i.e. ,  speaking-listening)  and 
cognitive-code  (i.e.,  inductive  and  deductive)  techniques  are  used  to 
accommodate  varying  student  learning  styles.  Material  is  normally  presented 
in  authentic  contexts  at  normal  conversational  speeds.  The  cultural  and 
background  Information  of  the  language  is  integrated  into  the  course  by  using 
various  audio-visual  training  aids.  The  curriculum  becomes  increasingly  more 
proficiency-based  and  less  achievement-oriented  over  the  length  of  the  course. 

Flexible-Scheduling  Instructional  Methodology.  The  flexible-scheduling 
method  was  similar  to  the  standard  DLIFLC  methodology.  (It  is  not  currently 
used  in  the  format  described  in  this  study. )  The  major  difference  in  design 
is  that  the  former  used  a  pacing  of  the  presentation  materials  based  upon  the 
instructors'  assessment  of  the  group's  readiness  rather  than  a  fixed  schedule 
for  the  presentation  of  materials.  The  emphasis  of  the  instruction  was  on 
speaking  through  exercises  based  upon  the  PSI  approach.  The  weekly  training 
schedule  was  the  device  used  for  implementing  and  monitoring  the  pace.  For 
it's  development,  the  instructors  analyzed  the  value  and  function  of  each 
lesson  component  in  terms  of  the  group's  ability  to  grasp  and  utilize  the 
material  presented.  Daily  discussions  between  instructors  allowed  for 
schedule  changes  to  accommodate  advancement,  review,  or  remediation.  Student 
input  was  incorporated  into  the  schedule. 

Contrasts  Between  Methodologies.  The  physical  environment  for  the  groups 
was  different.  The  suggestopedia  group  had  larger  rooms  for  its  sections. 

Room  size  was  smaller  and  identical  for  the  DLIFLC  sections  and  was 
approximately  one-half  the  size  of  the  rooms  used  by  suggestopedia  sections. 
The  suggestopedia  group  had  swivel,  high-back  chairs  and  no  desks.  The  DLIFLC 
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groups  had  standard  wooden  chairs  and  worked  at  tables.  The  suggestopedia 
group  used  Baroque  and  contemporary  music.  No  music  was  used  with  the  DLIFLC 
groups. 

The  suggestopedia  group  was  not  given  homework  whereas  the  other  groups 
had  specific  homework  assignments.  Weekly  graded  exercises  were  used  with  the 
DLIFLC  groups  but  not  for  the  suggestopedia  group.  Module  textbooks  were 
provided  to  all  students  during  inprocessing  at  DLIFLC,  but  the  suggestopedia 
students  were  told  they  did  not  need  them  and  they  were  seldom  used  according 
to  self  reports. 

The  suggestopedia  group  had  one  instructor  per  section  whereas  the 
standard  DLIFLC  group  averaged  four  instructors  per  day,  with  one  teaching 
three  hours  and  considered  to  be  the  primary  instructor.  The  flexibly- 
scheduled  group  had  two.  instructors  teaching  an  equal  number  of  hours  for  the 
same  daily  six-hour  schedule  as  the  other  two  groups. 

Suggestopedia  emphasized  oral  practice  with  the  omission  of  reading  and 
writing  skills,  particularily  during  the  first  five-weeks.  The  standard  group 
attempted  an  instructional  balance  between  the  three  skills,  while  the 
flexibly-scheduled  group  attempted  the  same  balance  with  slightly  more 
emphasis  on  oral  practice. 

Other  distinctions  between  the  standard  DLI  instruction  and  suggestopedia 
were  not  as  clear.  This  may  be  attributed  to  inconsistencies  within  the 
methodologies  as  applied  in  the  classroom  as  well  as  to  similarities  among  the 
methodologies.  Mignault  (1978),  in  his  discussion  of  Lozanov's  methods  of 
suggestopedia,  compared  Lozanov's  methods  with  certain  critical  aspects  of 
language  instruction  and  found  that  suggestopedia  incorporated  techniques 
noted  elsewhere  but  which  have  apparently  not  been  consolidated  or  adapted  for 
use  in  a  unified  program  of  instruction.  Examples  of  overlap  include  student 
participant  dialogues  and  a  positive  approach  by  the  instructor(s) .  These 
examples  are  emphasized  by  the  suggestopedia  methodology  but  are  also  applied 
in  varying  degrees  by  the  two  other  methodologies. 

The  emphases  on  games  and  role-playing  were  more  characteristic  of  the 
suggestopedia  instructional  process  than  to  the  DLIFLC  instructional 
methodologies.  The  suggestopedia  instruction  was  further  distinguished  by  the 
teaching  of  patterning  of  information  through  intonation,  pitch,  rhythm,  and 
proper  breathing.  It  is  worth  notiug  that,  toward  the  end  of  the  study,  the 
suggestopedia  instructors  were  informed  that  Lozanov  had  reduced  the  emphasis 
on  these  characteristics.  This  was  not  considered  by  the  suggestopedia 
instructors  as  having  an  effect  on  findings  from  this  study. 

Similarities  Between  Methodologies.  The  importance  placed  by 
suggestopedia  on  the  authority  and  prestige  of  the  instructor  did  not  seem  to 
differ  from  the  two  DLIFLC  groups.  This  comparability  may  be  an  incidental 
consequence  of  the  student  population  who,  as  military  personnel,  are  expected 
to  respond  to  teachers  as  figures  of  authority  and  prestige. 

All  groups  were  generally  presented  materials  in  authentic  contexts  and  at 
conversational  speed.  The  target  language  was  used  as  much  as  possible, 
though  more  so  with  the  suggestopedia  group,  particularly  during  the  first  two 
or  three  weeks. 
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All  groups  were  administered  and  evaluated  with  the  same  measures  of 
academic  performance  and  student  attitudes  with  the  exception  of  the  flexibly- 
scheduled  group.  This  group  was  not  initially  identified  for  the 
administration  of  attitudinal  measures  due  to  a  lack  of  certainty  about  its 
continuation,  and  incorporation  into  the  study.  The  effects  of  testing  and 
evaluation  on  the  suggestopedia  process  of  providing  a  less  stressful,  more 
relaxed  atmosphere  was  an  initial  concern.  However,  the  contractor 
representative  for  suggestopedia  and  the  Lozanov  Learning  Systems,  Inc.  (LLSI) 
felt  that  the  process  and  outcomes  would  not  be  adversely  affected 
(Schleicher,  1985).  Measures  that  addressed  proficiency  abilities  were  more 
acceptable  to  the  contractor  than  the  measures  of  achievement  because  of  the 
instructional  emphasis  by  suggestopedia  on  proficiency. 

It  is  important  to  note  at  this  point  that  certain  influences  on,  or 
modifications  to,  the  suggestopedia  methodology  at  DLIFLC,  such  as  the  effect 
of  testing  on  student  stress  cited  earlier,  were  not  considered  by  the  LLSI 
contract  representative  to  significantly  affect  achievement,  proficiency,  or 
attitudinal  outcomes.  Such  modifications  of  the  suggestopedia  methodology 
considered  acceptable  included  the  classroom  environment,  course  length,  the 
selection  process  of  student  and  instructor  personnel,  testing  procedures,  the 
degree  of  instructor  familiarity  with  the  testing  instruments,  and  concurrent 
military  duty  requirements  of  the  students  (Schleicher,  1985). 


The  purpose  of  this  study  was  to  compare  the  effectiveness  of  the 
suggestopedia  method  of  instruction  with  the  standard  DLIFLC  method  currently 
used.  An  additional  evalution  was  conducted  of  a  flexibly-scheduled 
methodology,  comparing  it  to  both  the  suggestopedia  and  standard  DLIFLC 
instruction. 

Results  from  the  comparisons  of  treatment  effectiveness  were  used,  in 
part,  to  determine  whether  the  treatments  could,  wholly  or  partially,  be 
acceptable  and/or  adaptable  for  utilization  within  Army  language  training 
programs. 

Tests  of  equality  among  groups  were  conducted  on  descriptive  variables 
considered  to  have  a  possible  effect  on  treatment  outcomes.  No  differences 
among  groups  were  expected  because  the  sample  was  drawn  from  a  generally 
homogeneous  population. 

Tests  of  significance  were  conducted  among  groups  on  results  from  measures 
of  academic  performance.  Further  analyses  were  conducted  as  necessary  for 
differences  between  groups  and  within  group  effect.  Performance  gains  for  the 
suggestopedia  group  after  10  weeks  of  language  training  were  expected  to  be 
the  same  as  the  performance  gains  for  the  flexibly-scheduled  and  standard 
DLIFLC  groups  after  14  and  15  weeks  respectively.  This  expectation  was  based 
upon  the  assertion  that  suggestopedia  could  accelerate  learning  (SSC  Contract 
No.  DABT-60-84-R-0080 ,  1984).  No  differences  were  expected  between  the 
flexibly-scheduled  and  standard  DLIFLC  groups  because  of  the  similarity  of 
their  methodologies  and  the  short  difference  in  time  of  one  week  between  dates 
for  the  completion  of  the  curriculum. 
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The  suggestopedia  group's  highest  spores  were  expected  to  be  on  the  oral 
communicative  skills  measured,  while  the  DLIFLC  groups  were  expected  to  have 
comparable  scores  on  all  measures  of  academic  performance.  These  expectations 
were  derived  from  suggestopedia's  emphasis  on  oral  skills  and  the  DLIFLC 
groups'  instructional  balance  on  each  skill  tested. 

An  analysis  of  variance  was  expected  to  find  significant  differences  in 
student  attitudes  among  groups.  Student  attitudes  from  the  suggestopedia 
group  were  expected  to  be  more  positive  then  those  found  for  the  DLIFLC  groups 
because  of  the  emphasis  placed  upon  strong  positive  student  attitudes  by  the 
suggestopedia  method.  No  differences  were  expected  between  the  DLIFLC  groups 
because  of  the  similarities  between  the  instructional  methodologies, 
specifically  the  use  of  the  Progressive  Skill  Integration  (PSI)  approach. 
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METHOD 


Subjects 

The  study  included  forty  junior  enlisted  Army  personnel  scheduled  to  begin 
the  Russian  Basic  Course  (RBC)  randomly  selected  and  sorted  into  two  sections 
each  for  the  suggestopedia  (n»20)  and  standard  DLIFLC  (n=*20)  groups.  One 
section  of  ten  junior  enlisted  Army  and  Navy  personnel  comprised  the  flexibly- 
scheduled  group.  This  group  was  previously  identified  and  in  place  prior  to 
its'  incorporation  into  this  study. 

Design 

The  study  design  used  matched  assignments  to  the  suggestopedia  and 
standard  DLIFLC  methodologies.  The  following  descriptive  variables  were  used 
to  match  the  two  groups:  military  rank,  military  occupational  specialty  (MOS), 
age,  years  of  military  service,  Defense  Language  Aptitude  Battery  (DLAB) 
scores,  educational  level,  prior  language  training,  General  Technical  (GT) 
scores,  and  gender.  Matching  was  not  possible  with  the  flexibly-scheduled 
group  because  of  its  later  addition  to  the  study. 

The  independent  variables  were  the  instructional  methodologies.  The 
dependent  variables  were  the  measures  of  academic  performance  and  student 
attitudes  toward  their  respective  methodologies. 

Instrumentation 

Effectiveness  was  measured  by  academic  performance  and  student  attitudes 
toward  their  instructional  methdologies. 

Three  measures  of  academic  performance  were  used.  One  was  a  set  of 
achievement  or  module  tests.  The  achievement  tests  had  two  major 
components:  written  and  oral.  An  example  of  each  is  provided  in  Appendixes  A 
and  B,  respectively.  Achievement  tests  are  designed  to  measure  performance  on 
materials  presented  over  the  course  of  the  study  (fifteen  weeks).  Five 
achievement  tests  were  administered  with  each  covering  a  particular  set  of 
materials  presented  over  different  lengths  of  time.  The  sets  of  materials  and 
allotted  time  for  presentation  were  called  modules.  The  modules  varied,  for 
example  Module  V  had  more  material  and  required  more  time  for  presentation 
than  any  of  the  previous  four  modules.  The  groups  completed  the  five  modules 
according  to  the  respective  schedules  (i.e. ,  the  suggestopedia  group  completed 
the  five  modules  in  ten  weeks,  the  flexibly-scheduled  group  in  fourteen  weeks, 
and  the  standard  DLIFLC  group  in  fifteen  weeks). 

A  second  measure  of  academic  performance  used  was  the  Proficiency 
Advancement  Test  (PAT),  a  combined  measure  of  both  achievement  and 
proficiency.  The  distinction  between  achievement  and  proficiency  is  that 
achievement  measures  performance  on  course  materials,  while  proficiency 
measures  performance  with  the  target  language  regardless  of  the  course  of 
instruction.  The  PAT  had  three  components:  listening,  reading,  and 
speaking.  An  example  of  each  is  provided  in  Appendixes  C,  D,  and  E, 
respectively. 
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The  third  measure  of  academic  performance  used  was  a  face-to-face  oral 
interview.  This  was  added  to  the  study  to  obtain  a  measure  of  conversational 
proficiency  only,  which  most  closely  approximates  suggestopedia's 
instructional  emphasis.  The  method  used  a  single  blind  technique  in  that  the 
two  rating  instructors  did  not  know  which  of  the  three  methodologies  the  rated 
students  experienced. 

The  measures  of  achievement  only,  achievement  and  proficiency,  and 
proficiency  only  were  used  in  order  to  provide  for  a  more  balanced  and 
comprehensive  evaluation  of  the  three  methodologies  than  would  be  available  by 
looking  at  only  achievement  or  proficiency. 

Student  attitudes  toward  their  methodologies  were  measured  using  weekly 
attitude  surveys,  a  pre-  and  posttest  questionnaire,  and  an  end-of-course 
questionnaire. 

The  weekly  attitude  survey  used  a  24-item  semantic  differential  scale.  It 
measured  student  attitudes  about  themselves  while  in  class,  their  opinions 
about  the  class,  and  opinions  about  their  instruetor(s).  A  sample  of  this 
instrument  is  provided  as  Appendix  F. 

The  questionnaire  administered  at  the  beginning  and  end  of  the  study,  the 
pre  and  posttest,  was  a  26-item  Likert-type  scale  derived  from  the  work  of 
Gardner  (personal  communication,  November  1,  1984).  It  addressed  student 
attitudes  about  the  potential  advantages  of  learning  Russian  (Part  A),  and 
student  attitudes  about  foreign  languages  in  general  (Part  B).  This 
instrument  is  provided  as  Appendix  G.  (Further  reading  on  Gardner's 
attitude/motivational  scales  as  related  to  language  learning  may  be  found  in 
Gardner,  [1983];  Gardner,  Clement,  Smythe,  &  Smythe,  [1979];  Gardner  &  Lalonde 
[1983];  and  Gliksman,  Smythe,  &  Gardner  [1982].) 

The  end-of-course  questionnaire,  as  provided  in  Appendix  H,  was  a 
developmental  instrument,  also  based  upon  the  work  of  Gardner,  with  126  items 
measuring  12  attitudinal  topics  using  a  variety  of  scale  types  (i.e.,  Likert, 
semantic-differential,  and  multiple-choice).  The  scales  included  measures  of 
attitudes  toward  the  learning  context,  student  motivation  and  anxiety,  and" 
attitudes  designed  to  reflect  some  key  characteristics  of  suggestopedia.  The 
topics  and  associated  scales  are  described  in  Table  H-l. 

Procedure 

The  study  of  the  various  instructional  methodologies  was  conducted  in  a 
resident  foreign  language  training  environment  at  the  DLIFLC,  Monterey, 
California. 

Prior  to  the  study  the  students  received  an  orientation  from  the  Director 
of  the  School  of  Russian,  DLIFLC,  outlining  the  study.  A  copy  of  the 
orientation  is  provided  as  Appendix  I.  In  addition,  an  informal  briefing  was 
provided  to  the  experimental  groups,  suggestopedia  and  flexibly-scheduled, 
which  stressed  that  there  would  be  no  negative  effects  as  a  result  of  any 
substandard  academic  performance  during  the  course  of  the  study. 

Administration  of  the  instruments  consisted  of  distributing  the  attitude 
questionnaires,  reading  the  privacy  act  statement  and  the  standardized 


instructions,  and  collecting  the  completed  questionnaires.  At  the  beginnning 
of  the  study,  students  were  administered  the  pretest  questionnaire.  The 
weekly  attitudinal  survey  was  administered  at  the  end  of  each  week  during 
language  laboratory  classes  or  during  breaks  between  classes.  All  instructors 
delayed  the  start  of  a  class  if  a  student  needed  more  time  to  complete  a 
survey.  Measures  of  academic  performance  were  part  of  the  regular  class 
schedule.  Results  were  returned  after  all  groups  had  completed  testing  with 
the  same  instrument.  The  posttest  and  end-of-course  questionnaires  were 
administered  during  language  laboratories  at  the  conclusion  of  the  study 
according  to  the  completion  of  the  Term  I  (five  modules)  curriculum  for  each 
of  the  methodologies;  the  suggestopedia  and  standard  DLIFLC  methodologies 
projected  completion  dates  were  weeks  10  and  15  respectively;  the  flexibly- 
scheduled  group  found  ihat  they  finished  Term  I  in  14  weeks. 

An  additional  assessment  of  student  attitudes  was  conducted  by  the  DLIFLC 
Office  of  Organizational  Effectiveness  at  the  conclusion  of  the  study  and  is 
available  as  a  supplementary  report  (Edwards,  1985). 
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RESULTS 


Descriptive  Variables 

Tests  of  equality  found  no  significant  differences  on  descriptive 
variables  considered  to  have  a  potential  effect  on  treatment  outcomes.  The 
variables  examined  are  as  follows:  military  rank,  military  occupational 
specialty  (MOS),  age,  years  of  military  service,  Defense  Language  Aptitude 
Battery  (DLAB)  scores,  educational  level,  prior  language  training,  General 
Technical  (GT)  scores,  gender,  and  a  pretest  measure  of  student  attitudes 
about  foreign  languages  in  general  and  the  potential  advantages  of  learning 
Russian.  (A  comparison  between  pretest  and  posttest  results  may  be  found  in 
the  section  titled,  "Measures  of  Student  Attitudes".) 

The  GT  score  is  considered  an  approximate  measure  of  both  verbal  ability 
and  arithmetic  reasoning.  Because  of  the  particularly  close  relationship 
between  verbal  ability  and  language  learning,  tests  of  group' equivalency  were 
conducted  on  both  components.  An  analysis  of  variance  (ANOVA)  found  no 
significant  differences  among  groups  on  either  measure. 

The  DLAB  and  GT  scores  are  measures  of  two  criteria  used  for  accepting 
personnel  into  the  residence  language  training  program  at  the  DLIFLC. 
Therefore  they  were  isolated  for  comparison  with  the  1984  student 
population.  Tests  of  equality  found  the  study  sample  to  be  representative  of 
the  1984  Russian  Basic  Course  Army  enlisted  student  population  on  both  DLAB 
and  GT  scores. 


Measures  of  Academic  Performance 


Achievement  Measures 


Achievement  or  module  tests  comprised  one  of  the  three  types  of  measures 
of  academic  performance.  A  one-way  analysis  of  variance  was  conducted  on  the 
two  components  of  the  achievement  tests,  written  and  oral.  An  additional 
ANOVA  was  conducted  on  the  weighted  exam  average.  The  weighted  exam  average 
was  the  higher  value  assigned,  by  the  DLIFLC,  to  the  written  component  as 
compared  to  the  oral  component  when  averaging  both  scores  together.  (Since 
the  completion  of  this  study,  the  DLIFLC  has  been  in  the  process  of  changing 
this  emphasis  on  the  written  component  toward  an  equal  weighting  between 
both.)  The  suggestopedia  group  had  significantly  lower  scores  than  the  two 
DLIFLC  groups  on  the  written  and  oral  components,  and  the  exam  averages. 

There  were  no  differences  between  the  two  DLIFLC  groups. 

The  number  of  students  available  for  the  administration  of  a  given 
instrument  or  component  thereof  was  subject  to  student  availability.  Student 
availability  was  affected  by  such  factors  as  medical  problems  and  changes  in 
military  duty  assignments.  Therefore,  the  number  of  cases  for  a  statistical 
test  varies. 

Written  Scores.  One  of  the  two  main  components  of  the  achievement  tests 
was  the  written  component.  Scores  are  presented  as  percentages  with  a 
possible  range  of  0-100.  As  expected  the  treatment  effect  was  significantly 
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different  among  groups  on  the  written  component  of  the  achievement  tests, 
£(2,46)  =  48.21,  £<.001.  Subsequent  t-tests  found  that  the  suggestopedia 
(iM  *  44.63)  group  did  significantly  worse  than  the  standard  DLIFLC  (M_  =  81.49) 
group,  £(38)  *  -8.65,  £<.001.  They  also  scored  significantly  lower  than  the 
flexibly-scheduled  (M  *  80.87)  group,  jt_( 27 )  *  6.79,  £<.001.  There  were  no 
significant  differences  between  the  bio  DLIFLC  groups,  t( 27)  -  -0.14,  £>.05. 
Subsequent  analyses  of  variance  between  sections  on  the  written  scores 
indicated  no  within  group  effect.  Appendix  B  provides  a  sample  of  the  written 
component  of  the  achievement  tests. 

Oral  Scores.  The  other  main  component  of  the  achievement  tests  was  the 
oral  score.  The  ANOVA  conducted  on  the  three  groups  indicated  a  significant 
difference  among  groups,  £(2,46)  *  11.43,  £<.001.  T-tests  indicated 
significantly  lower  scores  for  the  suggestopedia  (£  »  62.90)  group  when 
compared  to  the  standard  DLIFLC  (£  *  78.23)  group,  £(38)  *  3.94,  £<.001.  The 
suggestopedia  group  also  scored  significantly  lower  than  the  flexibly- 
scheduled  (£  *  79.93)  group,  £(27)  *  3.90,  £<.01.  No  differences  were  found 
between  the  DLIFLC  groups,  standard  and  flexibly-scheduled,  £(27)  =  0.40, 
£>.05.  The  analyses  of  variances  for  within  group  effect  resulted  in  no 
significant  differences.  Appendix  C  provides  a  sample  of  the  oral  portion  of 
the  achievement  tests. 

Exam  Average.  The  treatment  effect  was  significant  among  groups,  £(2,46) 

*  41.50,  £<.001.  Subsequent  t-tests  found  that  the  suggestopedia  (M  =  48.19) 
group  scored  signficantly  lower  than  the  standard  DLIFLC  (£  =  80.83)  group, 
£(38)  *  -7.96,  £<.001.  The  suggestopedia  group  also  had  significantly  lower 
scores  than  the  flexibly-scheduled  (£  =  80.68)  group,  £(27)  *  6.47,  £<.001. 

No  significant  difference  was  found  between  the  standard  DLIFLC  and  flexibly- 
scheduled  groups,  £(27)  =  0.973,  £>.05. 

A  further  analysis  of  variance  was  conducted  using  Hartley's  test  to 
determine  whether  the  variability  of  scores  between  sections  may  have  caused 
group  differences.  There  were  no  significant  differences  between  the  two 
sections  of  the  standard  DLIFLC  control  group,  E  and  F,  or  between  the  two 
sections  of  the  suggestopedia  group,  I  and  J. 

The  results  in  Table  1  reflect  the  significantly  lower  scores  for  the 
suggestopedia  group  as  compared  to  the  DLIFLC  groups.  Sections  are  listed  for 
comparisons  of  similarity  of  scores  within  groups.  The  table  also  reflects 
the  comparability  between  the  DLIFLC  groups  (standard  and  flexibly-scheduled) 
i  the  weighted  exam  average. 
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Table  l 


Means  and  Standard  Deviations  of  Module  Exam 
Averages  by  Section,  and  by  Group 


M 

SD 

Section  E  (standard  DLIFLC) 

84.67 

9.53 

Section  F  (standard  DLIFLC) 

76.99 

13.45 

Standard  DLIFLC  group  (avg.  of  sect.  E  and  F) 

80.83 

12.01 

Section  I  (suggestopedia) 

45.62 

10.22 

Section  J  (suggestopedia) 

50.76 

16.90 

Suggestopedia  group  (avg.  of  sect.  I  and  J) 

48.19 

13.85 

Flexibly-scheduled  Group  (one  sect.) 

80.68 

8.55 

Summary  of  Findings.  The  suggestopedia  group  scored  significantly  lower 
then  the  two  DLIFLC  groups  on  the  written  and  oral  components  of  the 
achievement  tests.  The  results  in  Table  2  indicate  that  the  greatest 
difference  between  the  suggestopedia  and  DLIFLC  groups  was  on  the  written 
portion  of  the  achievement  tests.  This  finding  was  expected,  in  part,  because 
of  the  emphasis  placed  by  the  suggestopedia  methodology  on  the  oral  component 
of  language  learning.  The  comparability  of  written  and  oral  scores  for  both 
the  standard  DLIFLC  and  f lexibly-scheduled  groups  reflects  the  intent  of  both 
methodologies  to  provide  an  instructional  balance  between  the  written  and  oral 
language  skills. 


Table  2 

Means  and  Standard  Deviations  of  Written 

and  Oral  Module  Results  by  Section,  and  by  Group 


Written  Results 

Oral 

Results 

M 

SD 

M 

SD 

Section  E 

85.30 

10.59 

82.12 

6.47 

Section  F 

77.68 

13.02 

74.34 

15.50 

Standard  DLIFLC  gp. 

81.49 

12.19 

78.23 

12.23 

Section  I 

41.92 

10.63 

60.80 

9.72 

Section  J 

47.33 

18.00 

64.99 

14.84 

suggestopedia  gp. 

44.63 

14.66 

62.90 

12.40 

F le  xi bly-s  chedu led 

80.87 

9.28 

79.93 

5.82 

group  (one  section) 
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Table  3  shows  the  results  of  the  five  achievement  or  module  tests  by 
component  and  exam  average,  and  by  group.  It  is  interesting  to  note  that  the 
two  DUFLC  groups  had  overall  higher  scores  on  the  written  measures  as 
compared  to  the  oral  measures  of  achievement.  Sugggestopedia  results  were  the 
opposite,  with  higher  oral  than  written  scores  which  is  again  in  consonance 
with  their  emphasis  on  the  oral  skills  of  language  learning. 


Table  3 


Means  and  Standard  Deviations  of  Module  I-V  Scores  by  Group 


MOD  Component (s) 

Standard 

DLIFLC 

Suggestopedia 

Flexibly-Scheduled 

M 

J5D 

M 

SD 

M 

SD 

MOD  I  written 

84.85 

12.93 

52.85 

14.38 

86.89 

6.45 

oral 

84.80 

13.21 

76.15 

8.89 

•  86.33 

4.00 

exam  avg. 

84.83 

12.53 

57.55 

12.53 

86,78 

5.63 

MOD  II  written 

81.79 

10.84 

40.20 

15.80 

79.78 

10.90 

oral 

78.21 

8.99 

66.95 

14.76 

81.33 

7.62 

exam  avg. 

81.09 

9.97 

45.55 

15.11 

80.09 

9.73 

MOD  III  written 

83.95 

8.50 

44.32 

15.51 

80.89 

10.47 

oral 

79.16 

8.03 

52.74 

21.45 

77.00 

9.00 

exam  avg, 

.82.99 

7.77 

45.47 

15.45 

80.11 

9.96 

MOD  IV  written 

80.84 

10.66 

44.95 

15.71 

70.56 

11.18 

oral 

77.26 

8.08 

55.79 

15.48 

76.67 

10.79 

exam  avg. 

80.13 

9.79 

47.05 

15.01 

78.98 

10.82 

MOD  V  written 

84.39 

8.89 

43.00 

14.76 

77.22 

14.30 

oral 

81.33 

7.87 

61.53 

18.33 

78.33 

4.92 

exam  avg. 

83.78 

8.31 

46.74 

14.56 

77.44 

12.10 

Combined  Achievement  and  Proficiency  Measures 

A  second  measure  of  academic  performance  used  was  the  Proficiency 
Advancement  Test  (PAT)  I.  The  PAT  was  a  combined  measure  of  both  achievement 
and  proficiency  with  emphasis  on  achievement.  An  analysis  of  variance  was 
conducted  to  compare  scores  among  groups  on  the  three  components  of  the  PAT: 
listening,  reading,  and  speaking.  The  suggestopedia  group  had  significantly 
lower  scores  than  the  two  DLIFLC  groups  on  the  reading  and  speaking 
components.  There  were  no  significant  differences  among  the  three  groups  on 
the  listening  co  ponent.  There  were  no  significant  differences  between  the 
DLIFLC  groups  on  the  three  components. 


Listening  Scores.  No  significant  difference  was  found  among  the 
suggestopedia  ( £  =  58.46),  standard  DLIFLC  (M  =  66.23),  and  flexibly-scheduled 
(£  *  63.73)  groups. 

Reading  Scores.  Significant  differences  were  found  on  the  reading  scores 
among  the  three  groups,  £(2,43)  ■  6.16,  £<.01,  Subsequent  analyses  indicated 
a  significant  difference  between  the  suggestopedia  (£  *  66.71)  and  standard 
DLIFLC  (£  *  73.47)  groups,  £(35)  *  -2.93,  £<.01.  A  significant  effect  was 
also  found  between  the  suggestopedia  and  flexibly-scheduled  (£  *  75.10) 
groups,  £(26)  *  2.71,  £<.05.  No  differences  were  found  between  the  standard 
DLIFLC  and  flexibly-scheduled  groups,  £(25)  =  0.63,  £>.05. 

An  analysis  of  variance  between  the  sections  of  the  suggestopedia  group 
indicated  no  significant  differences.  There  were  significant  differences 
between  section  E  (£  *  77.06)  and  section  F  (£  =  69.89)  of  the  standard  DLIFLC 
group  £(16)  *  -3.13,  £<.01.  However,  an  analysis  of  variance  for  the  groups 
within  treatments  design  indicated  that  the  sections  did  not  significantly 
contribute  to  the  variability  between  groups. 

Speaking  Scores  The  ANOVA  found  a  significant  effect  among  groups  on  the 
speaking  scores  of  the  PAT  I,  £(2,43)  =  7.63,  £<.01. 

T-tests  between  groups  found  that  the  suggestopedia  ( £  =  .64)  group  scored 
significantly  lower  than  the  standard  DLIFLC  (£  =  .89)  group,  £(35)  =  -3.77, 
£<.01.  Scores  between  the  sections  within  each  group  were  comparable.  No 
significant  differences  were  found  between  the  suggestopedia  and  flexibly- 
scheduled  (M  *  .79)  groups,  £(26)  =  1.71,  £>.05.  Again  there  were  no 
significant  differences  between  the  standard  DLIFLC  and  flexibly-scheduled 
groups,  £(25)  *  1.53,  £>.05. 

The  range  of  scores  for  the  speaking  results  (0-1)  and  the  oral  interview 
scores  (0-1)  is  provided  in  Appendix  J  along  with  the  description  for  each 
range  (level).  Also  provided  in  Appendix  J  is  the  rating  expected  for  a 
language  basic  course  graduate  at  the  DLIFLC. 

Proficiency  Measure 

Face-to-Face  Oral  Interview  Scores.  The  oral  interview  was  the  third 
measure  of  academic  performance.  It  is  considered  to  be  a  measure  of 
conversational  proficiency  only. 

The  analysis  of  variance  of  the  oral  interview  scores  indicated  no 
differences  among  groups.  These  results  indicate  that  the  suggestopedia  group 
had  the  same  level  of  proficiency  after  week  10  that  the  DLIFLC  groups  had 
after  14  and  15  weeks  of  study. 

Summary  of  Findings 

As  measures  of  proficiency  were  added  to  the  tests  of  academic 
performance,  differences  between  the  suggestopedia  and  the  DLIFLC  groups  were 
reduced.  This  is  demonstrated  by  a  comparison  of  differences  between  the 
suggestopedia  and  DLIFLC  groups  on  the  PAT  results  (listening,  reading, 
speaking)  and  the  similarity  between  groups  on  the  results  from  the  oral 
interviews  as  shown  in  Table  4.  The  reduction  in  differences  between  groups 


as  measures  of  proficiency  were  increased  is  further  evidenced  by  a  comparison 
of  the  scores  from  Table  4  with  the  achievement  scores  from  Table  2. 

(However,  the  similarity  of  oral  interview  scores  among  groups  is  most 
probably  attributable  to  the  low  level  of  proficiency  attained  after  only  ten 
or  fifteen  weeks  of  study  as  well  as  the  earlier  emphasis  suggestopedia  places 
upon  proficiency  as  compared  to  the  more  gradual  development  of  proficiency  by 
the  two  DLIFLC  methodologies.)  Table  4  also  reflects  higher  reading  than 
listening  scores  for  all  three  groups. 


Table  4 

Means  and  Standard  Deviations  of  PAT  Scores  Listening, 
Reading,  and  Speaking),  and  Face-to-Face  Oral 
Interview  Scores  by  Group 


Standard 

DLIFLC 

F lexi bly-S  chedu led 

M 

SD 

M 

SD 

M 

SD 

Listening 

66.23 

8.57 

58.46 

11.44 

63.73 

9.82 

Reading 

73.47 

5.99 

66.71 

7.89 

75.10 

7.11 

Speaking 

.894 

.170 

.637 

.239 

.789 

.169 

Oral  Interview 

.650 

.165 

.674 

.262 

.644 

.167 

Measures 

of  Student 

Attitudes 

Overall,  the  attitudinal  instruments  failed  to  discriminate  among 
groups.  Exceptions  are  noted  in  the  following  sections  on  the  weekly  attitude 
survey  and  end-of-course  questionnaire.  Results  were  generally  positive 
across  groups  on  each  of  the  three  attitude  measurement  instruments:  weekly 
attitude  survey,  pre-  and  posttest  questionnaire,  and  end-of-course 
questionnai re. 

A  Z  score  variable  transformation  of  individual  student  scores  was  used  to 
standardize  results  from  the  instruments  with  different  observed  scales.  This 
procedure  involved  scales  from  the  weekly  attitude  survey  (weeks  1  thru  10), 
the  posttest,  and  the  end-of-course  questionnaire.  An  analysis  of  variance 
found  no  significant  differences  between  the  suggestopedia  and  standard  DLIFLC 
groups  on  a  measure  combining  all  attitudinal  results,  F(l,38)  =  0.45, 

£>.05.  An  ANOVA  which  included  the  attitudinal  measures  available  from  the 
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flexibly-scheduled  group  (i.e. ,  the  posttest  and  end-of-course  questionnaire) 
also  resulted  in  no  significant  differences  among  groups,  £(2,46)  =  0.64, 
£>.05. 

Weekly  Attitude  Surveys.  A  review  of  the  results  from  the  weekly  attitude 
survey  indicated  no  differences  between  the  suggestopedia  and  standard  DL1FLC 
groups  for  weeks  one  through  ten  on  measures  of  student  attitudes  about 
themselves  while  in  class,  and  student  opinions  about  the  class.  There  were 
no  differences  between  the  groups  on  student  opinions  about  their 
instructor(s)  for  weeks  five  through  ten.  However,  for  weeks  one  through  four 
student  opinions  from  the  suggestopedia  group  about  their  instructor(s)  tended 
to  be  more  positive  than  those  from  the  standard  DLIFLC  group. 

Pre-  and  Posttest  Questionnaire.  Results  of  t-tests  found  no  significant 
differences  between  pre  and  posttest  scores  for  either  the  suggestopedia  or 
standard  DLIFLC  group.  A  comparison  of  posttest  scores  between  the 
suggestopedia  and  standard  DLIFLC  groups  also  indicated  no  significant 
differences. 

End-of~Course  Questionnaire.  The  three  groups  were  compared  on  results 
from  four  scales  designed  to  measure  attitudes  toward  key  characteristics 
alleged  to  typify  suggestopedia  instruction:  concentration,  student- 
centeredness,  method,  and  effort  required.  There  were  no  differences  among 
groups  on  the  concentration,  student-cente redness,  and  method  scales. 

However,  there  was  a  difference  among  groups  on  the  effort  scale  with  the 
suggestopedia  group  indicating  a  lower  level  of  effort  required  in  their  study 
of  the  target  language  than  the  (self-reported)  effort  required  of  the  two 
DLIFLC  groups.  Results  were  comparable  between  the  DLIFLC  groups. 

Two  scales  designed  to  measure  anxiety  were  also  evaluated  for  group 
differences.  A  (Russian)  class  anxiety  scale  showed  no  overall  differences 
between  groups.  On  a  (Russian)  language  use  anxiety  scale,  the  suggestopedia 
group  indicated  greater  confidence  in  their  ability  to  use  the  Russian 
language  than  did  the  two  DLIFLC  groups. 

There  were  no  end-of-course  differences  among  groups  on  scales  measuring 
student  attitudes  toward  learning  Russian,  motivational  intensity,  desire  to 
learn  Russian,  attitudes  toward  their  instructor(s) ,  and  attitudes  toward 
their  (Russian)  course. 

On  the  results  from  the  scale  measuring  attitudes  toward  the  course 
materials,  the  flexibly-scheduled  and  standard  DLIFLC  groups  were  more 
positive  toward  their  course  materials  than  the  suggestopedia  group. 

Student  Comments.  The  importance  of  the  teacher  was  clearly  demonstrated 
across  groups  by  consistently  positive  student  comments.  Comments  from  week 
to  week  addressed  the  "friendly",  "comfortable",  and  "positive"  approach  of 
the  instructor(s).  The  instructor(s)  were  often  singled  out  by  name  for 
praise  by  the  students,  "....was  especially  helpful  on  verbs",  "....is  never 
boring".  During  one  period  of  instructor  absence,  the  students  noted  that  the 
quality  of  the  substitute  was  much  lower  and  they  wished  they  had  their 
instructor  back  as  quickly  as  possible  "so  that  we  could  learn  more". 


Student  comments  about  the  methodologies  were  also  noteworthy.  During  the 
first  four  or  five  weeks  student  comments  from  the  suggestopedia  group  were 
positive  and  included  remarks  such  as,  "I  like  it",  "learning  faster  and  with 
confidence",  and  "activities  are  creative  and  enjoyable".  After  the  first 
four  or  five  weeks  the  positive  attitudes  of  the  suggestopedia  group  were  not 
as  consistently  high  as  they  had  been.  Self-esteem  began  to  diminish  and 
students  began  to  want  more  cousework  structure.  At  weeks  nine  and  ten  of  the 
study,  when  the  suggestopedia  group  was  completing  their  course  of 
instruction,  student  comments  became  more  concerned  with  their  possible 
shortcomings  in  language  skill  areas  such  as  "writing"  and  "grammar". 

The  standard  DLIFLC  group  comments  during  the  first  several  weeks  were 
more  oriented  toward  their  initial  discomfort  with  the  intensity  of  the 
language  training,  they  felt  "nervous", "uptight", "flustered",  and  a  dislike 
for  the  use  of  "too  many  instructors".  As  noted  in  the  "Contrasts  Between 
Methodologies"  section,  the  suggestopedia  group  had  one  teacher  per  section 
whereas  the  standard  DLIFLC  sections  used  an  average  of  four  instructors  per 
day  with  the  primary  instructor  teaching  three  of  the  six  hours.  The 
flexibly-scheduled  group,  not  addressed  in  this  part  of  the  report,  used  only 
two  instructors  in  a  team  teaching  approach. 

During  the  course  of  the  study  the  standard  DLIFLC  group  became  more 
"comfortable",  "relaxed",  and  had  more  confidence  in  their  instruction,  as 
indicated  through  the  students'  perception  of  their  ability,  and  a  more 
positive  attitude  toward  their  instructors.  After  the  fourth  or  fifth  week, 
comments  centered  around  materials  considered  difficult  for  the  students,  such 
as  "perfect  and  imperfect  verbs". 

As  a  target  of  opportunity  not  originally  considered  in  the  design  of  the 
study,  the  flexibly-scheduled  group's  comments  were  only  solicited  after  week 
ten.  (It  should  be  noted  that  their  academic  performance  records  were 
available  and  eventually  used  for  evaluation  in  the  study.)  These  comments 
were  consistently  positive  for  the  five  weeks  they  were  measured,  weeks  11 
thru  15,  especially  toward  their  methodology  and  instructors.  Representative 
comments  consistently  found  among  the  ten  students  and  across  the  five  weeks 
were  as  follows: 

"I  feel  that  working  in  a  self-paced  group  has  been 
an  immense  help  to  me  and  that  because  of  the 
latitude  given  to  the  teachers  in  order  to  adopt 
[adapt]  their  lessons  to  us,  that  we  were  able  to  do 
better  than  we  could  have  otherwise." 

"I'm  very  certain  the  best  results  in  teaching  us 
Russian  is  to  keep  us  as  a  self-paced  group  and  even 
change  the  rest  of  the  department  to  that  way  of 
teaching.  We're  more  comfortable  this  way  and  our 
teachers  know  each  of  our  weak  points  whereas  a 
different  teacher  every  hour  won't  even  know  our 
names." 

"( . )  is  one  of  the  best  teachers  I  have  ever 

had,  she  always  tries  different  methods  to  help  us 
learn,  and  is  never  bored  with  teaching. 
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"( . )  is  very  good  too  and  is  very  knowledgeable 

about  the  customs  and  country." 

Summary  of  Findings.  The  results  indicated  that  there  were  no  attitudinal 
differences  among  groups,  except  as  previously  noted,  during  the  first  four 
weeks  of  attitudes  about  instructors  on  the  weekly  attitude  survey,  and  on 
three  scales  used  in  the  end-of-course  questionnaire.  The  similarity  of 
altitudes  across  groups  may  be  a  result  of  their  military  background  and 
having  the  same  daily  military  requirements.  Differences  among  groups  may 
al«o  have  been  attenuated  because  of  the  similarities  in  methodologies  among 
groups  as  discussed  in  the  section  titled,  "Description  of  Instructional 
Treatments".  The  short  timeframe  of  the  study  may  have  also  precluded  more 
differences  from  developing. 

The  positive  attitudes  across  groups  may  have  been  a  direct  and/or  latent 
result  of  the  positive  attitudes  of  the  instructors.  The  instructors' 
altitudes  may  have  been  positive,  in  turn,  not  so  much  as  an  effect  of  their 
methodology  but  rather  the  special  attention  received  from  the  administrative 
staff  and  outside  observers.  This  special  effect  would  be  more  likely 
{sustained  over  the  shorter  10-15  weeks  of  the  study  rather  than  the  total  47 
week  course  of  instruction.  Also,  better  instructors,  who  may  naturally 
engender  positive  student  attitudes,  may  have  been  selected  to  employ  the 
various  methodologies.  The  likelihood  of  positive  instructor  attitudes 
existing  and  affecting  the  students  is  enhanced  by  the  fact  that  the 
suggestopedia  and  flexibly-scheduled  instructors  were  also  shareholders  in  the 
instruction  because  of  their  involvement  in  the  creation  and  implementation  of 
their  respective  methodology. 

There  were  two  points  of  particular  interest  regarding  attitudes  of  the 
suggestopedia  group.  One  was  that  the  gradual  shift  away  from  the  high 
positive  attitudes  encountered  during  the  first  four  or  five  weeks  may  have 
been  a  result  of  instructor  "burnout"  with  the  suggestopedia  methodology  in  an 
intensive  learning  environment,  as  well  as  a  result  of  receiving  low  scores  on 
the  first  achievement  test.  Student  expectations  of  a  more  formal  student- 
teacher  relationship  may  have  had  an  impact  on  their  attitudes  as  a  result  of 
the  actual  more  informal,  relaxed,  and  positive  approach  of  the  instructor. 

As  previously  noted  in  the  "Method"  chapter,  students  in  the  suggestopedia 
group  were  told  that  grades  acquired  while  in  the  experiment  would  not 
adversely  effect  promotion  or  class  standing.  However,  it  is  uncertain  what 
real  effect  this  had  on  the  students.  The  second  point  worth  noting  is  that 
while  suggestopedia  student  attitudes  were  not  as  uniformly  positive  during 
the  later  weeks  of  the  study  they  did  remain  positive  while  receiving 
generally  low  academic  performance  scores.  The  standard  DLIFLC  and  flexibly- 
scheduled  groups  had  the  same  level  of  positive  attitudes  but  also  had 
generally  higher  academic  performance  scores. 

The  differences  among  groups  at  the  end  of  the  course  may  indicate  that 
suggestopedia's  emphasis  on  the  oral  skills  does  have  a  positive  effect  on 
students'  comfort  with  oral  communication  while  the  positive  attitudes  of  the 
DLIFLC  groups  may  be  tempered  by  their  concerns  with  the  other  language  skills 
(i.e.,  listening,  reading,  and  speaking).  The  students'  opinions  from  the 
suggestopedia  group  about  the  lack  of  effort  required  is  most  probably  a 
result  of  no  requirement  to  use  textbooks  and  no  homework  requirements.  This 
"lack  of  effort"  perception  needs  to  remain  in  context  with  their  overall 
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poorer  academic  performance  results  as  compared  to  results  from  the  two  other 
groups. 

The  more  positive  attitudes  of  the  flexibly-scheduled  and  DLIFLC  groups 
toward  their  course  materials  may  be  a  result  of  the  DLIFLC  groups  having 
take-home  materials  while  the  suggestopedia  group  had  none. 


DLIFLC  Office  of  Organizational  Effectiveness  (OE)  Assessment  of  Student 
Attitudes 

An  assessment  was  conducted  on  student  attitudes  and  their  impressions  in 
the  use  of  their  respective  methodologies.  The  format  involved  one  or  two 
interviewers  for  groups  of  students  ranging  from  5  to  12  students  per  group. 
The  overall  findings  were  as  follows: 

Students  experienced  high  energy  and  confidence  at 
the  beginning  of  the  course  [study].  As  the  course 
progressed  and  more  material  was  presented,  the 
confidence  level  rose  to  what  might  be  called  a 
"maintenance  level"  and  seemed  to  sustain  the 
student  the  remainder  of  the  course  [study]. 

The  major  learning  and  pedagogical  factors 
indicated  that  the  course  content  and  quality  of 
instruction  were  considered  to  be  excellent.  The 
suggestopedia  methodology  was  considered  to  be  a 
useful  means  of  [sic]  a  person  to  obtain  a 
speaking/ listening  competency  of  a  foreign  language 
in  a  short  period  of  time. 

The  learning  situation,  e.g.,  music,  soft  chairs, 
and  absence  of  distractions,  was  not  reported  by 
the  students  to  have  a  direct  effect  on  the 
enhancement  of  learning  a  foreign  language. 

These  conclusions  tend  to  support  the  findings  stated  for  the  attitudinal 
measurement  instruments,  especially  those  from  the  weekly  attitude  surveys. 
Further  detail  from  this  assessment  may  be  found  in  a  separate  report  by 
Edwards  (1985). 


DISCUSSION 


Based  upon  results  from  the  measures  of  academic  performance,  the 
suggestopedia  methodology  did  not  accelerate  learning  when  compared  to  the  two 
DLIFLC  groups.  In  fact,  the  comparison  of  gains  in  language  learning  among 
groups  found  that  those  of  the  suggestopedia  group  were  significantly  smaller 
than  those  of  the  two  DLIFLC  groups.  The  results  indicating  a  significant 
difference  were  from  both  the  written  £nd  oral  measures  of  the  achievement 
tests,  and  on  two  of  the  three  measures  (reading  and  speaking)  from  the 
Proficiency  Advancement  Test  (PAT). 

Though  the  suggestopedia  group  had  generally  lower  scores  than  the  DLIFLC 
groups  on  the  measures  of  academic  performance,  test  results  indicated  that 
there  were  smaller  differences  between  the  suggestopedia  and  DLIfLC  groups  as 
measures  of  proficiency  were  added  to  the  evaluation  of  student  performance. 
This  trend  is  noted  by  a  review  of  the  results  from  the  PAT  indicating  less 
differences  between  the  suggestopedia  and  DLIFLC  groups  than' were  found  on 
results  from  the  achievement  tests.  Additionally,  the  face-to-face  oral 
interview  findings,  reflecting  measures  of  proficiency  only,  showed  no 
differences  among  groups.  Therefore,  the  comparatively  lower  scores  of 
suggestopedia  found  on  achievement  measures  during  the  early  stages  of 
language  learning  when  suggestopedia  is  emphasizing  proficiency  and  the  DLIFLC 
groups  are  emphasizing  achievement  is  not  surprising.  However,  it  is 
important  to  note  that  evaluations  of  proficiency  during  the  early  stages  of 
language  learning  may  not  be  able  to  provide  clear  discriminations  between 
methods  of  instruction  because  of  the  low  level  of  proficiency  available  to 
the  students. 

The  combination  of  findings  from  this  study,  (i.e.,  to  include  the 
positive  student  attitudes  within  the  suggestopedia  group  despite  low  academic 
performance,  and  their  confidence  with  using  the  target  language),  along  with 
the  research  in  related  suggestopedia  areas,  indicate  a  potential  use  for  at 
least  some  of  the  components  found  in  the  suggestopedia  method.  Examples  of 
potentially  useful  components  of  suggestopedia  include  the  following:  the  use 
of  incidental  learning  as  a  teaching  technique;  the  emphasis  on  relaxation  and 
a  positive  attitude  toward  the  target  language;  and  the  attempt  to  immerse  the 
student  into  the  target  language. 


If  suggestopedia  were  to  be  used  as  some  form  of  enrichment  adjunct  to  the 
established  DLIFLC  or  other  military  language  training  programs,  it  should 
probably  be  used  from  one  to  five  weeks.  This  may  be  the  best  time  interval 
since  positive  student  attitudes  and  the  instructor  energy  required  to 
implement  a  suggestopedia  program  seemed  to  peak  and  then  diminish  after 
approximately  five  weeks  of  intensive  application.  The  effect  of  instructor 
energy  was  reported  by  the  suggestopedia  instructors  and  noted  in  the 
Schleicher  report  previously  cited.  Those  components  listed  in  the  previous 
paragraph  that  are  found  useful  could  be  incorporated  throughout  a  course  of 
instruction. 

The  evaluation  of  the  flexible-scheduling  methodology  indicates  that  there 
can  be  a  time  savings  over  a  course  of  instruction  by  the  instructor(s)"close 
monitoring  and  supervision  of  student  progression  through  a  curriculum.  The 
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time  saved  may  be  used  for  review,  for  the  presentation  of  new  material,  or 
for  early  advancement  to  additional  training  or  a  field  assignment.  Both  the 
results  from  measures  of  academic  performance  and  attitudes  indicated  that  the 
positive  findings  from  both  measures  were  sustained  over  the  course  of  the 
study.  In  fact,  an  informal  follow-up  evaluation  of  the  flexibly-scheduled 
group,  conducted  approximately  one  month  later  (to  determine  whether  the 
methodology  should  be  continued  throughout  the  47-week  course),  indicated  that 
they  had  maintained  their  high  levels  of  performance  and  positive  attitudes. 

This  report  is  based  upon  findings  which  are,  in  part,  a  result  of  a 
specific  experimental  design  characterized  by  such  elements  as  the  time 
covered  by  the  study,  the  population  sampled,  the  sample  size,  differences  in 
instructors,  and  the  instruments  employed.  Conclusions  drawn  from  this  report 
should  not  be  generalized  outside  the  context  of  this  study. 
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APPENDIX  A 


PART  I 

Stimulus 

Task 

Options 

Content 

No.  of  Items 


PART  II 

Stimulus 
Task 
Content 
No.  of  Items 


PART  III 

Stimulus 

Task 

Content 
No.  of  Items 


WRITTEN  COMPONENT  (ACHIEVEMENT  TESTS) 


MODULE  1 


LISTENING  COMPREHENSION 

Twenty  Russian  sentences  spoken  on  tape. 

Select  correct  translation  of  word  or  phrases  in  each  sentence. 
Four  English  options. 

Vocabulary  and  grammar  from  Module  1. 

20 


DICTATION 

Russian  sentences  spoken  twice  on  tape. 

Write  the  words  that  have  been  omitted  on  student's  sheet. 
Vocabulary  and  grammar  from  Module  1. 

11 


TRANSLATION 


Written  English  sentences. 

Translate  in  writing  each  sentence  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 

Vocabulary  and  grammar  from  Module  1. 

10 


A- 1 


J-  V- 


MODULE  II 


PART  I  DICTATION 

Stimulus  Russian  sentences  spoken  twice  on  tape. 

Task  Write  down  the  sentences  in  Russian. 

Content  Vocabulary  and  grammar  from  Modules  I  and  II 

(emphasis  on  Mod.  II). 

No.  of  Items  7 


PART  II  LISTENING  COMPREHENSION 

Two  short  Russian  passages  spoken  on  tape  once. 

Answer  in  English  in  writing  questions  which  are  written  in 
student's  copy  of  the  exam. 

Simple  passages  based  on  the  vocabulary  and  grammar  of  Modules  I 
and  II  (emphasis  on  Mod.  II). 
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PART  III  TRANSLATION 

Stimulus  English  sentences  written  on  student's  sheet. 

Task  Translate  in  writing  each  sentence  into  Russian. 

Use  of  lexical  aids  is  not  permitted. 

Content  Vocabulary  and  grammar  from  Modules  I  and  II. 

No.  of  Items  10 


Stimulus 

Task 

Content 

No.  of  Items 


A-2 


MODULE  III 


PART  I  DICTATION 

Stimulus  Russian  sentences  spoken  twice  on  tape. 

Task  Write  down  the  sentences  in  Russian. 

Content  Vocabulary  and  grammar  from  Modules  I-III 

(emphasis  on  Mod.  III). 

No.  of  Items  10 


PART  II  LISTENING  COMPREHENSION 

Two  short  Russian  passages  spoken  on  tape  once. 

Answer  in  English  in  writing  questions  which  are  written 
in  student's  copy  of  the  exam. 

Simple  passages  based  on  the  vocabulary  and  grammar  of 
Modules  I-III  (emphasis  on  Mod.  III). 

9 


PART  III  TRANSLATION 

English  sentences  written  on  student's  sheet. 
Translate  in  writing  each  sentence  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 

Vocabulary  and  grammar  from  Modules  I-III 
(emphasis  on  Mod.  III). 

9 


Stimuli 

Task 

Content 

No.  of  Items 


Stimulus 

Task 

Content 

No.  of  Items 


MODULE  IV 


PART  I 

Stimulus 

Task 

Content 

No.  of  Items 

PART  II 

Stimulus 

Task 

Content 

No.  of  Items 

PART  III 

Stimulus 

Task 

Content 
No.  of  Items 

PART  IV 

Stimulus 

Task 

Content 

No.  of  Items 


DICTATION 

Russian  sentences  spoken  twice  on  tape. 
Write  down  the  sentences  in  Russian. 
Vocabulary  and  grammar  from  Modules  I-IV 
(emphasis  on  Mod.  IV). 

10 


LISTENING  COMPREHENSION 

Two  short  Russian  passages  spoken  on  tape  once. 

Answer  in  English  in  writing  questions  which  are 
written  in  student's  copy  of  the  exam. 

Simple  passages  based  on  the  vocabulary  and  grammar  of 
Modules  I-IV  (emphasis  on  Mod.  IV). 

7 


FILL-INS 

English  words  in  Russian  sentences. 
Translate  in  writing  each  word  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 
Vocabulary  and  grammar  from  Module  IV. 

12 


TRANSLATION 

Written  English  sentences. 

Translate  in  writing  each  sentence  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 

Vocabulary  and  grammar  from  Modules  I-IV 
(emphasis  on  Mod.  IV). 

10 


MODULE  V 


PART  I 

Stimulus 

Task 

Content 

No.  of  Items 


PART  II 

Stimulus 

Task 

Options 

Content 

No.  of  Items 


PART  III 

Stimulus 

Task 

Content 
No.  of  Items 


PART  IV 

Stimulus 

Task 

Content 


DICTATION 

Russian  sentences  spoken  twice  on  tape. 
Write  down  the  sentences  in  Russian. 
Vocabulary  and  grammar  from  Modules  I-V 
(emphasis  on  Mod  V). 

6 


LISTENING  COMPREHENSION 

Short  statements  in  Russian  spoken  on  tape. 

Select  appropriate  response  to  fit  the  situation. 
Three  English  options. 

Simple  situations  involving  statements  based  on 
Modules  I-V  (emphasis  on  Mod.  V). 

5 


FILL-INS 

English  words  in  Russian  sentences. 
Translate  in  writing  each  word  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 
Vocabulary  and  grammar  from  Module  V. 

8 


TRANSLATION 

Written  English  sentences. 

Translate  in  writing  each  sentence  into  Russian. 
Use  of  lexical  aids  is  not  permitted. 

Vocabulary  and  grammar  from  Modules  I-V 
(emphasis  on  Mod.  V). 

9 


No.  of  Items 


APPENDIX  B 


ORAL  COMPONENT  (ACHIEVEMENT  TESTS) 
MODULE  1 


PART  I  READING 


Stimulus 

Task 

Content 


No.  of  Items 


Short  passage  in  Russian  (35  words). 

Read  the  text  aloud. 

Simple  Russian  text  based  on  the  familiar  and  unfamiliar 
vocabulary. 

I 


PART  II  SPEAKING 

Ten  information  questions  asked  in  Russian  by  the  instructor. 
The  questions  are  based  on  the  material  from  Module  1. 


MODULE  II 


PART  I  READING 

Stimulus  Short  passage  in  Russian  (about  40  words). 

Task  Read  the  text  aloud. 

Content  Simple  Russian  text  based  on  familiar  and  unfamiliar 

vocabulary. 

No.  of  Items  1 


PART  II  QUESTIONS 

Ten  information  questions  asked  in  Russian  by  the  instructor. 
The  questions  are  based  on  the  material  from  Module  II. 


PART  III  TASK 

One  task  written  in  English  to  elicit  an  oral  response  based  on  the  material 
of  Module  II. 


MODULE  III 


PART  I  QUESTIONS 

Ten  information  questions  asked  in  Russian  by  the  instructor. 
The  questions  are  based  on  the  material  from  Module  III. 


PART  II  TASKS 

Two  role-playing  situations  written  in  English  to  elicit  an  oral  response 
based  on  the  material  of  Module  III. 


1J-3 


MODULE  IV 


PART  I  QUESTIONS 

Ten  information  questions  asked  in  Russian  by  the  instructor. 
The  questions  are  based  on  the  material  from  Module  IV. 


PART  II  TASK 

One  task  written  in  English  to  elicit  an  oral  response  based  on  the  material 
of  Module  IV. 


MODULE  V 


PART  I  QUESTIONS 

Ten  information  questions  asked  in  Russian  by  the  instructor. 
The  questions  are  based  on  the  material  from  Module  V. 


PART  II  TASK 

One  role-playing  situation  written  in  English  to  elicit  an  oral  response  based 
on  the  material  of  Module  V. 


APPENDIX  C 


LISTENING  COMPREHENSION,  PROFICIENCY  ADVANCEMENT  TEST  (I) 


1.  PART  ONE  (WORDS  AND  PHRASES) 

Stimulus  -  Sentences/phrases  spoken  on  tape. 

Task  -  Select  correct  translation  of  word  or  phrases  in  each  level. 

Options  -  Four  English  options. 

Content  -  Familiar  high  frequency  expressions,  frozen  phrases. 

No.  of  Items  -  25 
Level  -  0+/1 

2.  PART  TWO  (SITUATIONAL  RESPONSE) 

Stimulus  -  Sentences  spoken  on  tape. 

Task  -  Select  appropriate  response  to  fit  situation. 

Options  -  Four  target  language  options. 

Content  -  Simple  situations  involving  very  simple  statements  and 
responses. 

No.  of  Items  -  15 
Level  -  0+/ 1 

3.  PART  THREE  (GISTING/SHORT  PASSAGE) 

Stimulus  -  Sentences  or  short  paragraphs  spoken  on  tape  twice. 

Task  -  Select  correct  summary  of  passage. 

Options  -  Four  English  options. 

Content  -  Simple  passages  reduced  to  Level  1+/2  wording. 

Words  in  multiple  choice  options  will  help  furnish  context;  glosses  in 
English  may  be  used  if  absolutely  necessary. 

No.  of  Items  -  15 
Level  -  1+/2 

4.  PART  FOUR  (FACTUAL  QUESTIONS /LONGER  PASSAGES) 

Stimulus  -  Four  or  five  passages  (50-70  words)  spoken  on  tape  twice. 

Task  -  Answer  six  or  seven  factural  multiple-choice  items  on  each  passage. 
Options  -  Four  English  options. 

Content  -  Simple  passages  reduced  to  Level  2  wording  with  very  minor 
editing;  words  in  multiple  choice  options  will  help  furnish  context; 
glosses  may  be  used  if  absolutely  necessary. 

No.  of  Items  -  23-30 
Level  -  2 


Total  No.  of  Items  -  85 
Range  -  0+/2 


APPENDIX  D 


READING  COMPREHENSION,  PROFICIENCY  ADVANCEMENT  TEST  (I) 


1.  PART  ONE  -  (SIGNS  AND  EXPRESSIONS) 

Stimulus  -  Printed  signs,  frozen/memorized  phrases. 

Options  -  Four  English  options. 

Number  of  Items  -  10 
Level  -  0+ 

2.  PART  TWO  (VOCABULARY  IN  CONTEXT) 

Stimulus  -  Short  printed  sentences  with  underlined  word. 

Task  -  Choose  translation  of  underlined  words. 

Options  -  Four  English  options. 

Context  -  High  frequency/familiar  vocabulary  in  familiar' context. 

No.  of  Items  -  15 
Level  -  0+/1 

3.  PART  THREE  (CONTEXTUAL  COMPREHENSION) 

Stimulus  -  Four  short  sentences  or  paragraphs  with  blanks. 

Task  -  Choose  correct  form  to  fill  blank. 

Options  -  Four  target  language  options. 

Content  -  High  t requency/familiar  grammar  patterns  and  forms. 

No.  of  Items  -  15 
Level  -  0+/1 

4.  PART  FOUR  (QUESTIONS  ON  PASSAGE /GI STING) 

Stimulus  -  Two/three  short  passages  (45-70  words). 

Task  -  Answer  multiple-choice  questions  (factual  summary). 

Options  -  Four  English  options. 

Content  -  Authentic*  passages  reduced  to  Level  1+/2  wording. 
Multiple-choice  options  will  help  furnish  context:  glosses  in  English  may 
be  used  if  absolutely  necessary  to  furnish  context. 

No.  of  Items  -  12 
Level  -  1/1+ 


*(In  these  documents  the  word  "authentic"  refers  to  published  written  material 
in  the  target  language  outside  the  course  of  instruction.) 


5. 


PART  FIVE  (CLOZE  PASSAGE) 


Stimulus  -  Short  passage  (80-120  words)  with  paraphrase. 

Task  -  Select  options  to  restore  passage. 

Options  -  3-4  symbols,  12-16  deletions,  10-15  options  under  each  symbol. 
Content  -  Simple  authentic  passage  with  very  little  editing;  paraphrase 
will  be  used  to  furnish  context. 

Glosses  in  English  may  be  used  if  absolutely  necessary  to  furnish  context 
No.  of  Items  -  16 
Level  -  1+/2 

6.  PART  SIX  (INFORMATION  IDENTIFICATION) 

Stimulus  -  Short  passage  (125-150  words)  with  underlined  words  and 
phrases. 

Task  -  Select  multiple-choice  option  that  translates  underlined  portion. 
Options  -  4  multiple-choice. 

Content  -  Simple  authentic  passage  with  very  little  editing;  glosses  in 
English  may  be  used  if  absolutely  necessary  to  furnish  context. 

No.  of  Items  -  17 
Level  -  1+/2 


Total  No.  of  Items  -  85 
Level  Range  -  0+/2 


!)-2 


APPENDIX  E 

SPEAKING  COMPONENT,  PROFICIENCY  ADVANCEMENT  TEST  (I) 

PART  ONE 

Three  information  questions  designed  to  elicit  speech  at  levels  0+ 
through  1. 

PART  TWO 

Two  role-playing  situations  designed  to  elicit  speech  at  levels  0+ 
through  1, 


APPENDIX  F 


WEEKLY  ATTITUDE  SURVEY  INSTRUMENT  name 

Privacy  Act  Statement  _ 

SSAN 

You  are  being  asked  to  complete  this  questionnaire  as  part  of  a  research 
effort  which  seeks  to  find  ways  of  improving  both  teaching  and  learning  at 
DLIFLC.  Disclosure  of  requested  information  is  solicited  under  the  authority 
of  Title  10,  United  States  Code  3012,  Executive  Order  9397 j  and  Army  Regulation 
600-2,  and  is  voluntary.  Failure  to  provide  name  or  SSAN  may  result  in 
misidentification  and  thus  jeopardize  the  findings  and  conclusions  of  this 
study,  with  possible  negative  impact  on  the  Defense  Foreign  Language  Program 
as  a  whole. 

Your  answers  to  all  questions  will  be  held  in  strict  confidence,  and  your 
name  and  SSAN  are  requested  only  so  that  your  answers  can  be  associated  with 
other  information  that  is  essential  to  this  research  effort  and  is  contained  in 
your  official  records  and  training  files.  Neither  your  instructors  nor  your 
superior  officers  nor  anyone  else  other  than  the  researchers  will  be  able  to 
associate  your  identity  with  your  responses  on  this  questionnaire,  and  even 
the  researchers  will  use  assigned  numbers  to  refer  to  your  questionnaire  data 
and  to  all  other  information  that  is  collected  as  part  of  this  research. 

Student  responses  will  be  reported  only  as  aggregates  (e.g.,  "20$  of  the  students 
thought  that....")  or  as  anonymous  individuals  (e.g.,  "one  student  commented 
that ....)• 


In  order  for  the  results  to  be  meaningful,  it  is  important  that  your 
answers  be  accurate  and  frank  as  possible.  Also,  the  usefulness  of  your 
questionnaire  will  be  lessened  to  the  extent  that  you  do  not  answer  each  item. 
Therefore,  you  are  urged  to  answer  all  items  unless  it  is  very  important  to 
you  personally  to  omit  certain  ones,  in  which  case  you  may  omit  them  and  go  on 
to  the  others.  If  you  have  questions  about  any  items,  please  raise  your  hand 
for  assistance. 


F-l 


From  previous  experience  and  research  we  know  that  students  feelings  and 
reactions  to  a  class  and  to  instructors  change  over  time.  We  would  like  to 
know  how  your  feelings  change  as  this  class  progresses.  We  will  ask  you  to 
respond  to  the  items  on  the  next  page  each  week.  Only  the  researchers  will 
have  access  to  this  information;  it  will  not  be  used  as  any  part  of  the  official 
performance  appraisal  of  your  instructors.  So  please  be  completely  frank  and 
honest  in  your  responses  knowing  that  strict  confidentiality  will  be  maintained. 

For  each  pair  of  words  place  an  ”x”  in  the  space  that  best  reflects  what  you 
feel  and  how  strongly  you  feel  about  the  idea  conveyed  by  each  word-pair. 

Using  the  word-pairs  below,  describe  your  actual  feelings  while  in  this 
class  (that  is,  describe  yourself  while  in  class)  during  the  past  week: 

1  2  3  4  5  6  7 


comfortable 

• 

• 

• 

• 

uncomfortable 

attentive 

• 

• 

• 

• 

restless 

uncertain 

• 

• 

• 

confident 

positive 

• 

• 

• 

negative 

relaxed 

« 

• 

• 

• 

• 

• 

• 

• 

uptight 

trusting 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

guarded 

bored 

• 

• 

* 

• 

• 

* 

• 

• 

* 

• 

♦ 

• 

stimulated 

insignificant 

• 

• 

• 

♦ 

• 

• 

• 

• 

• 

• 

• 

• 

significant 

happy 

• 

• 

« 

• 

• 

• 

• 

• 

• 

• 

• 

• 

unhappy 

calm 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

flustered 

smart 

* 

• 

• 

• 

• 

• 

• 

• 

• 

♦ 

• 

• 

stupid 

depressed  __ 

♦ 

• 

• 

• 

« 

• 

• 

• 

• 

• 

• 

• 

excited 

Us«  the  following  set  of  word-pairs  to  describe  your  opinions  about  the 
class  during  th«  paat  weak* 


liked 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

a 

disliked 

interested 

• 

• 

• 

• 

• 

• 

a 

• 

• 

• 

• 

• 

bored 

easy 

a 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

hard 

dreaded 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

• 

enjoyed 

tedious 

« 

• 

• 

• 

• 

• 

• 

• 

« 

• 

• 

• 

fascinating 

positive 

• 

• 

a 

• 

• 

• 

• 

• 

• 

• 

• 

negative 

Use  the  following  word-pairs  to  describe  your  opinions  about  your 
instructor(s)  during  the  past  week. 


competent 

• 

a 

« 

« 

• 

a 

a 

• 

• 

a 

a 

a 

incompetent 

caring 

• 

• 

• 

a 

• 

• 

a 

• 

a 

a 

a 

a 

insensitive 

unsure 

a 

• 

« 

• 

• 

a 

a 

a 

• 

a 

a 

a 

confident 

dull 

• 

• 

• 

a 

• 

• 

a 

a 

• 

a 

exciting 

uncomfortable 

• 

• 

• 

a 

a 

• 

a 

• 

a 

a 

a 

a 

relaxed 

creative  _ 

• 

• 

• 

♦ 

a 

a 

• 

a 

a 

a 

a 

a 

uncreatlve 

Any  other  comments  you  would  like  to  sake? 


APPENDIX  G 


PRETEST  AND  POSTTEST  QUESTIONNAIRE  name 

Privacy  Act  Statement  _ 

SSAN 

You  are  being  asked  to  complete  this  questionnaire  as  part  of  a  research 
effort  which  seeks  to  find  ways  of  improving  both  teaching  and  learning  at 
DLIFLC.  Disclosure  of  requested  information  is  solicited  under  the  authority 
of  Title  10,  United  States  Code  3012,  Executive  Order  9397,  and  Army  Regulation 
600-2,  and  is  voluntary.  Failure  to  provide  name  or  SSAN  may  result  in 
misidentification  and  thus  jeopardize  the  findings  and  conclusions  of  this 
study,  with  possible  negative  impact  on  the  Defense  Foreign  Language  Program  as 
a  whole e 

Your  answers  to  all  questions  will  be  held  in  strict  confidence,  and  your 
name  and  SSAN  are  requested  only  so  that  your  answers  can  be  associated  with 
other  information  that  is  essential  to  this  research  effort  and  is  contained  in 
your  official  records  and  training  files.  Neither  your  instructors  nor  your 
superior  officers  nor  anyone  else  other  than  the  researchers  will  be  able  to 
associate  your  identity  with  your  responses  on  this  questionnaire,  and  even  the 
researchers  will  use  assigned  numbers  to  refer  to  your  questionnaire  data  and 
to  all  other  information  that  is  collected  as  part  of  this  research.  Student 
responses  will  be  reported  only  as  aggregates  (e.g.,  "20l  of  the  students 
thought  that....")  or  as  anonymous  individuals  (e.g.,  "one  student  commented 
that ...."). 

In  order  for  the  results  to  be  meaningful,  it  is  important  that  your 
answers  be  accurate  and  frank  as  possible.  Also,  the  usefulness  of  your 
questionnaire  will  be  lessened  to  the  extent  that  you  do  not  answer  each  item. 
Therefore,  you  are  urged  to  answer  all  items  unless  it  is  very  important  to  you 
personally  to  omit  certain  ones,  in  which  case  you  may  omit  them  and  go  on  to 
the  others.  If  you  have  questions  about  any  items,  please  raise  your  hand  for 
assistance. 


G-l 


PART  A 


The  following  statements  are  about  potential  advantages  of  learning  Russian. 
However,  you  may  not  agree  that  a  given  statement  has  any  relevance  to  you 
personally.  For  example : 

"I  like  having  the  opportunity  to  learn  Russian  because  I  will  be 
able  to  read  Russian  literature  in  the  original." 

If  reading  Russian  literature  is  totally  irrelevant  to  you,  and  you  can't 
imagine  why  anyone  would  want  to,  you  would  write  1_  for  strongly  disagree.  On 
the  other  hand  if  reading  Russian  literature  in  the  original  is  one  of  your 
most  important  reasons  for  learning  Russian,  you  would  write  6_  for  strongly 
agree.  Of  course,  your  response  to  this  statement  may  lie  somewhere  between 
those  two  extremes;  in  that  case,  you  would  write  2,  3,  4,  or  5* 

Agree  Agree  Agree  Disagree  Disagree  Disagree 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly 
6“  5^321 

I  like  having  the  opportunity  to  learn  Russian  because  it  will.... 

_  1.  help  me  get  a  Job  after  I  have  completed  my  military  service. 

_  2.  increase  my  ability  to  influence  others. 

_  3.  enable  me  to  better  understand  Russian  culture. 

_  4.  make  me  a  better  educated  person. 

_  5.  give  me  an  edge  in  competing  with  others. 

_  6.  advance  ray  military  career. 

_  7.  enable  me  to  get  to  know  Russians. 

_ ,  8.  help  me  understand  what  Russians  really  want. 

_  9*  make  me  appear  more  cultured. 

_  10.  enable  me  to  meet  and  converse  with  a  greater  variety  of  people. 

_  11.  help  me  earn  a  college  degree. 

_  12.  enable  me  to  interact  socially  with  Russians. 

_  13.  help  me  get  the  kind  of  job  I  want  in  the  military. 

_  lit.  help  me  protect  my  interests  in  dealing  with  Russians. 

_  15.  help  me  qualify  for  additional  training  in  the  military. 

_  16.  make  me  more  attractive  to  future  employers. 


PART  B 


This  part  of  the  questionnaire  asks  you  to  indicate  how  much  you  agree  or 
disagree  with  a  series  of  statements  about  foreign  languages  in  general.  Your 
opinions  or  feelings  may  lead  you  to  agree  with  some  statements  and  disagree 
with  others.  There  are  no  right  or  wrong  answers — Just  your  point  of  view. 

REMEMBER:  1.  It  is  your  honest  opinion  that  is  being  requested,  and 

your  responses  will  be  treated  with  Btrict  confidentiality. 

2.  Mark  each  statement  according  to  your  first  impression; 
it  is  not  necessary  to  take  a  lot  of  time  for  any  one 
question. 


Please  read  each  statement  carefully  and  write  in  the  number  that  best  describes 
your  response. 

Agree  Agree  Agree  Disagree  Disagree  Disagree 

Strongly  Moderately  Slightly  Slightly  Moderately  Strongly 
6  5^321 

_  1.  I  would  really  like  to  learn  many  languages. 


2.  When  I  see  a  foreign  film,  I  would  rather  hear  the  sound  track 
in  English  than  to  hear  the  original  language  and  see  English 
subtitles. 

3.  Foreign  languages  sound  like  gibberish  to  me. 


U.  Knowing  foreign  languages  can  help  one  convey  many  feelings  and 
ideas  that  are  not  easily  expressed  in  English. 

5.  I  wish  I  could  speak  several  languages  fluently. 


6.  If  I  planned  to  live  in  another  country,  and  I  thought  I  could 
get  along  in  English,  I  would  not  make  such  effort  to  learn  the 
language . 

7.  I  really  have  little  interest  in  foreign  languages. 

8.  I  often  wish  I  could  read  newspapers  and  magazines  in  many 
languages. 

9.  I  enjoy  meeting  and  listening  to  people  who  speak  other  languages. 


10.  Studying  a  foreign  language  is  not  a  pleasant  experience. 
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NAME 


DATE 

APPENDIX  H 

END-OF-COURSE  QUESTIONNAIRE 


Now  that  the  methods  comparison  experiment  in  which  you  have  been 
participating  has  come  to  a  close,  we  would  like  to  ask  you  to  help  us  evaluate 
it. 

You  are  being  asked  to  complete  this  questionnaire  as  part  of  a  research 
effort  which  seeks  to  find  ways  to  improving  both  teaching  and  learning  here  at 
DLI.  Your  answers  to  all  questions  will  be  treated  in  strict  confidence.  We 
ask  for  your  name  only  to  associate  your  responses  on  this  questionnaire  with 
other  information  of  importance  to  our  research.  We  will  immediately  assign 
numbers  to  your  questionnaire  and  all  other  information  we  collect  and  only  the 
numbers  will  be  used  in  our  analysis.  Neither  your  instructors  nor  anyone 
else  at  DLI  or  in  the  military  will  have  access  to  any  of  these  questionnaire 
responses  except  as  aggregated  in  research  findings  (e.g.,  "20^  of  the  students 
felt  that....).  The  researchers  will  be  the  only  persons  who  will  see  the 
questionnaire  with  your  name  on  it. 

For  the  results  to  be  meaningful,  it  is  important  that  you  be  as  accurate 
and  as  frank  as  possible  in  your  answers.  If  you  do  not  want  to  answer  any 
particular  item,  you  do  not  have  to.  However,  you  should  realize  that  the 
usefulness  of  your  questionnaire  will  be  lessened  to  the  extent  that  you  do  not 
answer  each  item.  We,  therefore,  urge  you  to  answer  all  items  unless  it  is 
important  to  you  personally  to  omit  certain  ones.  If  you  have  difficulties  or 
questions  about  any  one  of  the  items,  please  raise  your  hand  and  someone  will 
come  to  your  assistance. 

Thank  you  very  much  for  your  cooperation. 
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Section  I 


For  each  of  the  following  statements,  indicate  the  number  from  the  following 

scale  which  best  represents  your  reaction  to  the  statement. 

Disagree  Disagree  Disagree  Agree  Agree  Agree 

Strongly  _  Slightly  Slightly  _  Strongly 
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_  1.  I  enjoyed  participating  in  class. 

_  2.  I  found  that  the  course  required  extraordinary  effort  on  my  part. 

_  3*  My  mind  often  wandered  when  I  was  in  class. 

_  4.  If  I  had  it  to  do  over  again,  I  would  avoid  studying  Russian. 

_  5.  I  did  not  get  anxious  when  I  had  to  respond  in  class. 

_  6.  I  would  feel  calm  and  sure  of  myself  if  I  had  to  order  a  meal  in 

Russian. 

_  7.  I  would  have  preferred  another  method  of  teaching  this  course. 

_  8.  I  often  felt  uncomfortable  in  class. 

_  9*  If  ever  I  should  run  into  a  group  of  people  speaking  Russian, 

I  would  feel  relaxed  in  joining  them. 

_____  10.  I  found  the  instructors  responsive  to  my  particular  learning 

needs. 

_  11.  Most  things  we  learn  in  Russian  are  interesting. 

_  12.  I  really  couldn't  understand  people  who  got  uptight  about  using 

Russian  in  class. 

_  13.  It  was  easy  to  remain  attentive  in  this  class. 

_  14.  So  far,  I  have  found  this  program  easier  than  I  expected. 

_  15*  I  really  enjoy  learning  Russian. 

_  16.  I  would  feel  uncomfortable  speaking  Russian  in  any  real-world 

situation. 

_  17.  My  instructor(s)  seemed  genuinely  interested  in  my  progress  in 

this  course. 

_  18.  I  never  felt  quite  sure  of  myself  when  I  was  speaking  in  class. 

_  19.  I  learn  more  when  other  teaching  methods  are  used. 

_  20.  In  all  honesty,  I  would  rather  do  almost  anything  that  study 

Russian. 

_  21.  This  was  One  of  the  most  demanding  courses  I  have  ever  taken. 

_  22.  I  felt  confident  when  active  participation  took  place  in  class. 

_  23*  I  would  feel  comfortable  speaking  Russian  in  an  informal 

gathering  where  both  English  and  Russian  speaking  persons  were 
present. 

_  24.  I  plan  to  continue  my  study  of  Russian  after  I  complete  this 

course. 


Disagree 


Agree 


Disagree 

Strongly 

1 


Disagree 

Slightly 

3 


Agree 

Slightly 

1* 


Agree 

Strongly 

6 


25.  When  in  class,  I  was  completely  absorbed  in  what  was  going  on 
in  the  classroom. 

26.  To  be  honest,  I  hate  Russian. 

27.  My  instructors  were  adequately  sensitive  to  the  problems  of  a 
beginning  student. 


28.  I  often  dreaded  going  to  class. 

29.  I  am  sure  I  would  get  nervous  if  I  had  to  speak  Russian  to  a 
sales  clerk. 


30.  Learning  Russian  takes  so  long,  the  attempt  does  not  seem 
worthwhile. 


31.  It  embarrassed  me  to  volunteer  answers  in  class. 

32.  I  usually  had  plenty  of  time  to  complete  all  class  assignments. 

33 •  I  would  feel  confident  and  relaxed  if  I  had  to  ask  street 
directions  in  Russian. 

3^.  I  am  glad  to  have  the  opportunity  to  learn  Russian. 

35*  My  instructor(s)  seemed  determined  to  cover  specified  material 
regardless  of  student  readiness. 

36.  I  often  had  difficulty  keeping  my  attention  focused  on  class¬ 
room  activities. 

37-  I  think  learning  Russian  is  boring. 

38.  It  bothered  me  that  the  other  students  spoke  Russian  in  class 
better  than  I  did. 

39 •  I  would  get  flustered  if  it  were  necessary  to  speak  Russian 
when  making  a  telephone  call. 

1*0.  I  would  recommend  that  this  course  always  be  taught  using  this 
method. 

1*1.  Making  a  hotel  reservation  in  Russian  would  bother  me. 

1*2.  The  satisfaction  of  learning  Russian  makes  the  effort  worthwhile. 

1*3.  I  was  generally  relaxed  in  class. 

1*1*.  This  course  seemed  better  geared  to  the  way  I  learn  than  other 
courses  I  have  taken. 
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Section  II 


Please  answer  each  of  the  following  items  by  circling  the  letter  of  the 
completion  which  appears  to  be  most  applicable  to  you.  Please  be  as  frank  and 
accurate  as  possible;  the  proper  evaluation  of  this  experiment  depends  upon  it. 

45.  During  off-duty  hours,  I  thought  about  what  I  had  learned  in  my  Russian 
class .... 

a .  very  frequently 

b.  seldom  or  never 

c.  once  in  a  while 

46.  When  I  had  a  problem  understanding  something  we  were  learning  in  Russian 
class,  I  would  usually.... 

a.  ask  the  instructor  for  help 

b.  seek  help  only  just  before  the  exam 

c.  just  forget  about  it 

47.  When  working  on  assignments,  I  usually.... 

a.  put  some  effort  into  them,  but  not  as  much  as  I  could 

b.  worked  very  carefully,  making  sure  I  understand  everything 

c.  did  as  little  as  possible 

d.  [Not  applicable — we  had  no  assignments.] 

48.  Considering  how  I  studied  Russian,  I  can  honestly  say  that  I.... 

a.  did  as  much  work  as  the  next  person 

b.  will  pass  on  the  basis  of  sheer  luck  or  intelligence,  because  I  did 
very  little  work 

c.  worked  very  hard  to  learn  Russian 

d.  [Not  applicable — we  were  not  required  to  study  outside  of  class.] 

49.  If  my  instructor  wanted  someone  to  do  an  extra  Russian  assignment,  I 
would .... 

a.  definitely  not  volunteer 

b.  definitely  volunteer 

c.  do  it  only  if  the  instructor  asked  me  directly 

d.  [Not  applicable — we  had  no  assignments.] 

50.  After  I  got  my  Russian  assignments  back,  I  usually.... 

a.  reviewed  them,  correcting  my  mistakes 

b.  just  put  them  aside  and  went  on 

c.  looked  them  over,  but  didn't  bother  correcting  mistakes 

d.  [Not  applicable — we  had  no  assignments.] 

51.  When  I  was  in  Russian  class,  I.... 

a.  volunteered  answers  as  much  as  possible 

b.  answered  only  the  easier  questions 

c.  hardly  ever  said  anything 

52.  When  there  was  an  event  involving  Russian  language  or  culture,  I  usually 
went .... 

a.  even  if  I  had  to  pay 

b.  only  if  it  was  free  or  subsidized 

c.  only  if  required  to  do  so 
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53.  I  made  a  point  of  talking  to  my  instructors  outside  of  class.... 

a.  only  if  I  was  having  difficulty  understanding  something 

b.  to  learn  as  much  as  I  could 

c.  not  at  all 

54.  I  used  the  Russian  materials  in  the  Learning  Resources  Center  (films, 
tapes,  etc.).... 

a.  regularly 

b.  seldom  or  never 

c.  once  in  a  while 

55.  If  I  thought  by  staying  in  the  military  I  had  a  good  chance  to  take 
intermediate  and  advanced  Russian.... 

a.  I  would  definitely  go  for  it 

b.  it  would  make  no  difference  in  my  plans 

c.  I  would  get  out  as  soon  as  possible 

56.  I  want  to  learn  enough  Russian.... 

a.  to  get  through  the  course 

b.  to  be  really  fluent 

c.  to  do  my  job  satisfactorily 

57.  The  amount  of  Russian  I  have  learned  so  far.... 

a.  is  enough  to  turn  me  off  completely 

b.  is  satisfactory;  I  can  get  by  with  it 

c.  has  just  whetted  my  appetite  for  more 

58.  The  further  along  I  go  in  my  Russian  studies,  the  more  I.... 

a.  wish  I  had  started  long  ago 

b.  want  to  keep  on  going  with  it 

c.  wish  it  were  all  over 

59.  During  Russian  class,  I  would  like.... 

a.  to  have  a  combination  of  Russian  and  English  spoken 

b.  to  have  as  much  English  as  possible  spoken 
c«  to  have  only  Russian  spoken 

60.  If  I  had  the  opportunity  to  speak  Russian  outside  of  class,  I  would.... 

a.  never  speak  it 

b.  speak  Russian  most  of  the  time,  using  English  only  if  really  necessary 

c.  speak  it  occasionally 

61.  If  it  were  strictly  up  to  me  whether  or  not  to  learn  Russian,  I.... 

a.  would  definitely  take  it 

b.  would  not  take  it 

c.  don't  know  whether  I  would  take  it  or  not 

62.  I  find  studying  Russian.... 

a.  not  interesting  at  all 

b.  as  interesting  as  most  other  subjects  I've  studied 

c.  especially  interesting 
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63.  If  I  had  the  opportunity  I  would  watch  Russian  TV  and  listen  to  Russian 
radio. . . . 

a.  sometimes 

b.  as  often  as  possible 

c .  never 

64.  If  they  were  available,  I  would  read  Russian  magazines  and  newspapers.... 

a.  as  often  as  I  could 

b.  seldom  or  never 

c.  not  very  often 

Section  III 

The  scales  that  follow  attempt  to  capture  your  overall  impressions  of  the 
instructors,  course  materials,  and  course  in  general,  to  which  you  have  just 
been  exposed.  You  will  be  asked  to  rate  each  of  these  three  things  on  a  series 
of  scales,  each  of  which  consists  of  a  pair  of  words  expressing  opposites  with 
seven  spaces  in  between.  For  example,  assume  you  are  rating  the  course  on  this 
scale: 

12  3  4  5  6  7 

hard _ : _ : _ : _ : _ : _ : _ easy 


If  you  place  an  "x"  in  the  "1"  position,  it  means  that  you  found  this  course 
to  be  extremely  hard;  an  "x”  in  the  "7"  position  would  mean  you  found  it  ex¬ 
tremely  easy.  Positions  2-3  and  5-6  are  used  to  indicate  gradations  of  these 
opinions,  and  position  4  indicates  that  you  found  the  course  neither  hard  nor 
easy,  but  somewhere  in  the  ’’average"  range. 

In  answering  this  part  of  the  questionnaire,  please  work  quickly  and  do 
not  stop  to  think  about  each  scale.  It  is  your  immediate  impressions  in  which 
we  are  interested. 


A.  INSTRUCTOR(S) 

For  each  pair  of  words  on  this  scale,  place  an  ”x"  in  the  position  which 


best  fits  your  impression  of  your  primary  instructor! s) . 

1  2  3  4  5  6  7 

efficient  :::::: 

inefficient 

insensitive  :::::: 

sensitive 

cheerful  :::::: 

cheerless 

competent  :::::: 

incompetent 

insincere  :::::: 

sincere 

unapproachable  :::::: 

approachable 

pleasant  :::::: 

unpleasant 

trusting  :::::: 

suspicious 

incapable  :::::: 

capable 

tedious  :::::: 

fascinating 
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friendly 

exciting 

organized 

unreliable 

unimaginative 

impatient 

polite 

colorful 

unintelligent 

good 

industrious 

boring 

dependable 

disinterested 

inconsiderate 


unfriendly 

dull 

disorganized 

reliable 

imaginative 

patient 

impolite 

colorless 

intelligent 

bad 

unindustrious 

interesting 

undependable 

interested 

considerate 


B.  COURSE  MATERIALS 


For  each  pair  of  words  on  this  scale,  place  an  "x"  in  the  position  which 
best  fits  your  impression  of  the  course  materials  you  used. 

1  2  3  4  5  6  7 


important 

meaningless 

dull 

interesting 

organized 

valuable 

confusing 

hard 

natural 

irrelevant 

clear 

useful 


unimportant 

meaningful 

stimulating 

boring 

disorganized 

worthless 

coherent 

easy 

artificial 

relevant 

unintelligible 

useless 
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For  each  pair  of  words  on  this  scale,  place  an  "x"  in  the  position  which 
best  fits  your  impression  of  the  course  as  a  whole  as  you  experienced  it  during 
the  weeks  in  which  the  experiment  was  in  progress. 


meaningful 

enjoyable 

monotonous 

effortless 

awful 

interesting 

good 

simple 

disagreeable 

fascinating 

worthless 

necessary 

appealing 

useless 

elementary 

pleasurable 

educational 

unrewarding 

difficult 

satisfying 

unimportant 

pleasant 

exciting 

clear 

colorful 


1  2  3  4  5  6  7 

_ : _ : _ : _ : _ : _ : _  meaningless 

_ : _ : _ : _ : _ : _ : _  unenjoyable 

_ _ : _ : _ : _ : _ : _ : _  absorbing 

_ _ : _ s _ ; _ : _ : _ : _  hard 

_ : _ : _ : _ : _ : _ s _  nice 

_ : _ : _ : _ : _ : _ : _  boring 

_ : _ : _ : _ : _ : _ : _  bad 

_ _ : _ : _ : _ : _ : _ : _  complicated 

_ : _ : _ : _ : _ : _ : _  agreeable 

__ _ : _ : _ : _ : _ : _ : _  tedious 

_ : _ : _ : _ : _ : _ : _  valuable 

_ _ : _ : _ : _ : _ : _ : _  unnecessary 

_ : _ : _ : _ :  : _ s _  unappealing 

_ : _ : _ : _ : _ : _ s _  useful 

_ _ : _ s _ : _ : _ s _ : _  complex 

_ : _ : _ : _ s _ : _ : _  painful 

_ _ s _ : _ : _ : _ : _ : _  noneducational 

_ : _ : _ : _ : _ : _ : _  rewarding 

_ : _ : _ : _ : _ : _ : _  easy 

_ : _ : _ : _ : _ : _ : _  unsatisfying 

_ : _ : _ : _ s _ : _ : _  important 

_ :  ; _ : _ : _ : _ : _  unpleasant 

_ : _ : _ s _ : _ : _ : _  dull 

_ : _ : _ : _ : _ : _ : _  confusing 

:  : _ :  : _  :  :  colorless 


THANK  YOU  FOR  ALL  YOUR  COOPERATION 1 
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Table  H-l.  Description  of  the  End-of-Course  Questionnaire 


Attitude  Title 

Definition 

Item  No. 

Scale  Type 

Concentration* 

ability  to  sustain  focus  on,  or  not 
be  distracted  from  class-room  activity 

3,13,25,26 

Likert 

Student- 

Centeredness* 

students'  perception  that  they  are 
important  and  that  their  needs/interests 
are  relevent  to  the  instructors 
and  course  structure. 

10,17,27,35 

Likert 

Method* 

students'  assessment  of  the  method  of 
instruction 

7,19,40,44 

Likert 

Effort  Required* 

the  amount  of  time  and  energy 
(work)  students 

2,14,21,32 

Likert 

Russian  Class 
Anxiety 

anxiety  felt  by  students 
in  the  language  classroom 

1,2,3,12,22, 

28,31,38,43 

Likert 

Russian  Use 

Anxiety 

student  anxiety  toward  using 
the  language  outside  the  classroom 

6,9,16,23,29, 

33,39,41 

Likert 

Attitudes  Toward 
Learning  Russian** 

designed  to  assess  positive  or 
negative  feelings  about  learning  Russian 

4,11,15,20,24 

26,30,34,37,42 

,  Likert 

Motivational 

Intensity** 

an  assessment  of  how  hard  students  work 
to  learn  Russian 

45-54 

Multiple- 

Choice 

Desire  to  Learn 
Russian** 

designed  to  assess  strength  or  weakness 
of  feelings  about  learning  Russian 

55-64 

Multiple- 

Choice 

My  (Russian) 
Instructor 

one  of  two  scales  used  to  measure 
attitudes  toward  the  learning  situation 

Part  A  (25 
items) 

Semantic- 

Differential 

My  (Russian) 

Cou rse 

the  second  of  two  scales  measuring 
attitudes  toward  the  learning  situation 

Part  C  (25 
items) 

Semantic- 

Differential 

Course  Materials 

designed  to  measure  attitudes  toward 
the  course 

Part  B  (12 
items) 

Semantic- 

Differential 

*Four  key  characteristics  alleged  to  typify  suggestopedia  instruction 

**T‘nree  scales  used  to  measure  the  variable  called  "motivation" 
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SCRIPT  FOR  STUDENT  ORIENTATION  ON  EXPERIMENT 


Some  of  you  may  wonder  how  a  particular  method  of  teaching  or  training  is 
approved  and  authorized.  Since  DLI  is  responsible  for  conducting  language 
training,  developing  course  materials  for  use  here  and  in  the  field, 
developing  tests  for  use  here  and  throughout  DOD  (proficiency  tests  and 
aptitude  tests — DLPT/DLAB),  for  language  training  standards  and  for  research 
in  Foreign  Language  (FL)  training,  we  constantly  need  to  evaluate  these 
efforts  and  attempt  to  improve  them. 

Newly  developed  materials  and  tests,  for  example,  are  validated  by 
actually  using  them  with  student  groups,  then  by  analyzing  and  studying  the 
results.  Subsequently,  any  revisions  which  may  be  needed  are  made  in  an 
effort  to  continue  improving  our  courses  and  the  general  training  provided  at 
DLIFLC.  We  also  ask  students  for  their  opinions  and  attitudes  at  various  time 
through  questionnaires  and  interviews.  All  this  is  again  part  of  our 
continuing  effort  to  provide  the  best  language  training  possible  for  DLI 
students. 

The  DLI  staff,  in  cooperation  with  the  Army  Soldier  Support  Center  and  the 
Army  Research  Institute,  has  a  plan  to  try  out  two  kinds  of  methods  of 
teaching  Russian  with  your  class.  We  plan  to  try  these  out  with  four  of  the 
sections  of  this  class.  With  10  students  per  sections,  we  will  use  method  "A" 
in  two  sections  for  a  total  of  20  students.  With  two  other  sections  or  a 
total  of  20  students,  we  will  use  method  "B".  This  study  will  only  last 
during  the  first  term  of  the  Basic  Course  or  approximately  15  weeks.  At  that 
time  a  Proficiency  Advancement  Test  (PAT  I)  is  given  to  all  students  anyway. 
All  students  are  expected  to  pass  PAT  1  before  moving  on  to  Term  II. 

Both  groups  will  cover  essentially  the  same  amount  of  material  that  is 
covered  by  all  students  in  Term  I  of  the  Basic  Course,  except  that  there  will 
be  differences  in  what  goes  on  in  the  classroom.  The  learning  and  teaching  in 
two  of  these  sections  using  method  "A"  will  be  somewhat  different  than  the 
learning  and  teaching  in  the  two  sections  normally  used  in  DLI  classes.  Two 
other  sections  will  follow  the  normal  DU  pattern  of  teaching  and  learning 
(Method  "B")  except  that  records  kept  on  all  four  groups  will  be  analyzed  and 
compared  in  much  more  detail.  Also,  there  may  be  somewhat  more  attention 
given  to  observations  of  the  classroom  activities  and  to  the  progress 
individual  students  are  making.  There  will  not  be  any  extra  work  of  any  kind, 
so  the  same  amount  of  effort  will  be  expected  anyway  for  students  in  all 
sections. 

The  Army  is  fully  behind  this  study  and  expects  service  members  to  apply 
themselves  by  learning  Russian  as  their  basic  duty  regardless  of  which  section 
they  may  be  assigned  to.  Normal  assignments  to  sections  in  DLI  Basic  Courses 
is  controlled  by  the  language  department  chairpersons  and  their  supervisors. 

We  do  not  make  up  sections  artificially  by  putting  all  the  people  with  high 
scores  on  tests  such  as  DLAB  and/or  ASVAB  in  one  section  and  all  the  low 
scorers  in  another.  Rather  we  normally  seek  balanced  grouping  in  all 
sections,  so  there  is  no  special  advantage  in  being  in  one  section  over  being 


in  another  one.  For  example,  we  prefer  to  have  the  same  ratio  of  males  and 
females  in  all  sections. 

Once  you  are  assigned  to  one  of  these  four  sections,  you  are  expected  to 
remain  in  it  just  as  all  students  assigned  to  any  sections  are  expected  to 
remain  with  their  section  until  such  time  as  there  may  be  an  administrative 
reason  for  recombining  sections.  This  would  not  normally  happen  until  after 
15  weeks  anyway.  We  assume  that  you  will  put  your  best  efforts  into  learning 
Russian  regardless  of  which  section  you  are  assigned  to  and  without  favor  to 
one  supervisor  over  another  or  one  set  of  instructors  over  another. 

You  should  be  aware  that  proponents  for  method  "A!l  claim  that  people  learn 
much  faster  when  it  is  used.  This  is  not  necessarily  unusual,  since 
proponents  for  innovative  methods  almost  always  make  such  claims.  DLI's 
purpose  is  to  give  a  fair  chance  for  method  "A"  to  be  fully  evaluated  in 
comparison  with  DLI's  standard  method  of  teaching.  Incidentally,  DLI's 
standard  method  also  has  many  innovative  features,  especially  since  the  entire 
course,  including  materials  and  tests,  is  quite  new  and  is  still  subject  to 
some  revision.  So  just  consider  that  the  normal  mode  for  all  DLI  sections  is 
not  cast  in  concrete  but  rather  is  still  open  to  change  as  we  never  stop 
seeking  to  improve  all  of  our  training  materials  and  methods  so  that  our 
students  can  learn  more  effectively  and  efficiently. 

If  any  of  you  think  you  will  feel  uncomfortable  by  being  in  one  of  the 
four  sections  destined  to  be  given  more  evaluation,  please  say  so  now.  Once 
we  make  assignments  to  sections  we  do  not  expect  to  make  changes. 


Are  there  any  questions? 
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DESCRIPTIONS  AND  RATINGS 

Table  J-L.  Ratings  for  the  Speaking  Portion  of  the  PAT,  and  for  the 
(Face-to-Face)  Oral  Interviews 
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LANGUAGE  SKILL  LEVEL  DESCRIPTIONS 

FOR  SPEAKING  (PAT)  AND  ORAL  INTERVIEW  RATINGS 


Preface 

The  following  proficiency  level  descriptions 
characterize  spoken  language  use.  Each  of  the  six 
“base  levels”  (coded  00.  10,  20,  30.  40,  and  50) 
implies  control  of  any  previous  "base  level’s” 
functions  and  accuracy.  The  “plus  level” 
designation  (coded  06.  16,  26,  etc.)  will  be 
assigned  when  proficiency  substantially  exceeds 
one  base  skill  level  and  does  not  fully  meet  the 
criteria  for  the  next  “base  level.”  The  “plus  level” 
descriptions  are  therefore  supplementary  to  the 
"base  level"  descriptions. 

A  skill  level  is  assigned  to  a  person  through  an 
authorized  language  examination.  Examiners 
assign  a  level  on  a  variety  of  performance  criteria 
exemplified  in  the  descriptive  statements. 
Therefore,  the  examples  given  here  illustrate,  but 
do  not  exhaustively  describe,  either  the  skills  a 
person  may  possess  or  situations  in  which  he/she 
may  function  effectively. 

Statements  describing  accuracy  refer  to  typical 
stages  iri  the  development  of  competence  in  the 
most  commonly  taught  languages  in  formal 
training  programs  In  other  languages,  emerging 
competence  parallels  these  characterizations, 
but  often  with  different  details. 

Unless  otherwise  specified,  the  term  “native 
speaker"  refers  to  native  speakers  of  a  standard 
dialect. 

"Well-educated."  in  the  context  of  these 
proficiency  descriptions,  does  not  necessarily 
imply  formal  higher  education.  However,  in 
cultures  where  formal  higher  education  is 
common,  the  language-use  abilities  of  persons 
who  have  had  such  education  is  considered  the 
standard.  That  is.  such  a  person  meets 
contemporary  expectations  for  the  formal, 
careful  style  of  the  language,  as  well  as  a  range  of 
less  formal  varieties  of  the  language. 

Level  0  (No  Proficiency) 

Unable  to  function  in  the  spoken  language. 
Oral  production  is  limited  to  occasional  isolated 
words.  Has  essentially  no  communicative  ability. 
(Has  been  coded  S-0  in  some  nonautomated 
applications.)  [Computer  Code  00] 

Level  0+  (Memorized  Proficiency) 

Able  to  eatisfy  immediate  needs  using 
rehearsed  utterances.  Shows  little  real  autonomy 
of  expression,  flexibility,  or  spontaneity.  Can  ask 
questions  or  make  statements  with  reasonable 


accuracy  only  with  memorized  utterances  or 
formulae.  Attempts  at  creating  speech  are  usually 
unsuccessful. 

Examples:  The  individual's  vocabulary  is 
usually  limited  to  areas  of  immediate  survival 
needs.  Most  utterances  are  telegraphic;  that  is, 
functors  (linking  words,  markers,  and  the  like)  are 
omitted,  confused,  or  distorted.  An  individual  can 
usually  differentiate  most  significant  sounds 
when  produced  in  isolation,  but,  when  combined 
in  words  or  groups  of  words,  errors  may  be 
frequent.  Even  with  repetition,  communication  is 
severely  limited  even  with  people  used  to  dealing 
with  foreigners.  Stress,  intonation,  tone.  etc.  are 
usually  quite  faulty.  (Has  been  coded  S-0+  in 
some  nonautomated  applications.)  [Computer 
Code  06] 

Level  1  (Elementary  Proficiency) 

Able  to  satisfy  minimum  courtesy  requirements 
and  maintain  very  simple  face-to-face 
conversations  on  familiar  topics.  A  native  speaker 
must  often  use  slowed  speech,  repetition, 
paraphrase,  or  a  combination  of  these  to  be 
understood  by  this  individual.  Similarly,  the 
native  speaker  must  strain  and  employ  real-world 
knowledge  to  understand  even  simple 
statements/questions  from  this  individual.  This 
speaker  has  a  functional,  but  limited  proficiency. 
Misunderstandings  are  frequent,  but  the 
individual  is  able  to  ask  for  help  and  to  verify 
comprehension  of  native  speech  in  face-to-face 
interaction.  The  individual  is  unable  to  produce 
continuous  discourse  except  with  rehearsed 
material. 

Examples:  Structural  accuracy  is  likely  to  be 
random  or  severely  limited.  Time  concepts  are 
vague.  Vocabulary  is  inaccurate,  and  its  range  is 
very  narrow.  The  individual  often  speaks  with 
great  difficulty.  By  repeating,  such  speakers  can 
maxe  themselves  understood  to  native  speakers 
who  are  in  regular  contact  with  foreigners  but 
there  is  little  precision  in  the  information 
conveyed.  Needs,  experience,  or  training  may 
vary  greatly  from  individual  to  individual,  for 
example,  speakers  at  this  level  may  have 
encountered  quite  different  vocabulary  areas. 
However,  the  individual  can  typically  satisfy 
predictable,  simple,  personal  and  accommodation 
needs;  can  generally  meet  courtesy,  introduction, 
and  identification  requirements;  exchange 
greetings;  elicit  and  provide,  for  example, 
predictable  and  skeletal  biographical 
information.  He/she  might  give  information  about 
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business  hours,  explain  routine  procedures  in  a 
limited  way,  and  state  in  a  simple  manner  what 
actions  will  be  taken.  He/she  is  able  to  formulate 
some  questions  even  in  languages  with 
complicated  question  constructions.  Almost 
every  utterance  may  be  characterized  by 
structural  errors  and  errors  in  basic  grammatical 
relations.  Vocabulary  is  extremely  limited  and 
characteristically  does  not  include  modifiers. 
Pronunciation,  stress,  and  intonation  are 
generally  poor,  often  heavily  influenced  by 
another  language.  Use  of  structure  and 
vocabulary  is  highly  imprecise.  (Has  been  coded 
S-1  in  some  nonautomated  applications.) 
[Computer  Code  10] 

Level  1+  (Elementary  Proficiency,  Plus) 

Can  Initiate  and  maintain  predictable  face-tc- 
face  conversations  and  satisfy  limited  social 

demands.  He/she  may,  however,  have  little 
understanding  of  the  social  conventions  of 
conversation.  The  interlocutor  is  generally 
required  to  strain  and  employ  real-world 
knowledge  to  understand  even  some  simple 
speech.  The  speaker  at  this  level  may  hesitate  and 
may  have  to  change  subjects  due  to  lack  of 
language  resources.  Range  and  control  of  the 
language  are  limited.  Speech  largely  consists  of  a 
series  of  short,  discrete  utterances. 

Examples:  The  individual  is  able  to  satisfy  most 
travel  and  accommodation  needs  and  a  limited 
range  of  social  demands  beyond  exchange  of 
skeletal  biographic  information.  Speaking  ability 
may  extend  beyond  immediate  survival  needs. 
Accuracy  in  basic  grammatical  relations  is 
evident,  although  not- consistent.  May  exhibit  the 
more  common  forms  of  verb  tenses,  for  example, 
but  may  make  frequent  errors  in  formation  and 
selection.  While  some  structures  are  established, 
errors  occur  in  more  complex  patterns.  The 
individual  typically  cannot  sustain  coherent 
structures  In  longer  utterances  or  unfamiliar 
situations.  Ability  to  describe  and  give  precise 
information  is  limited.  Person,  space,  and  time 
references  are  often  used  incorrectly. 
Pronunciation  is  understandable  to  natives  used 
to  dealing  with  foreigners.  Can  combine  most 
significant  sounds  with  reasonable  comprehen¬ 
sibility,  but  has  difficulty  in  producing  certain 
sounds  In  certain  positions  or  in  certain 
combinations.  Speech  will  usually  be  labored. 
Frequently  has  to  repeat  utterances  to  be 
understood  by  the  general  public.  (Has  been 
coded  S-1+  in  some  nonautomated  applications.) 
[Computer  Code  16] 

Level  2  (Limited  Working  Proficiency) 

Able  to  satisfy  routine  social  demands  and 
limited  work  requirements.  Can  handle  routine 
work-related  interactions  that  are  limited  in 
scope.  In  more  complex  and  sophisticated  work- 


related  tasks,  language  usage  generally  disturbs 
the  native  speaker.  Can  handle  with  confidence, 
but  not  with  facility,  most  normal,  high-frequency 
social  conversational  situations  including 
extensive,  but  casual  conversations  about  current 
events,  as  well  as  work,  family,  and 
autobiographical  information.  The  individual  can 
get  the  gist  of  most  everyday  conversations  but 
has  some  difficulty  understanding  native 
speakers  in  situations  that  require  specialized  or 
sophisticated  knowledge.  The  individual’s 
utterances  are  minimally  cohesive.  Linguistic 
structure  is  usually  not  very  elaborate  and  not 
thoroughly  controlled;  errors  are  frequent. 
Vocabulary  use  is  appropriate  for  high-frequency 
utterances,  but  unusual  or  imprecise  elsewhere. 

Examples:  While  these  interactions  will  vary 
widely  from  individual  to  individual,  the  individual 
can  typically  ask  and  answer  predictable 
questions  in  the  workplace  and  give 
straightforward  instructions  to  subordinates. 
Additionally,  the  dividual  can  participate  in 
personal  and  accommodation-type  interactions 
with  elaboration  and  facility;  that  is,  can  give  and 
understand  complicated,  detailed,  and  extensive 
directions  and  make  non-routine  changes  in 
travel  and  accommodation  arrangements.  Simple 
structures  and  basic  grammatical  relations  are 
typically  controlled;  however,  there  are  areas  of 
weakness.  In  the  commonly  taught  languages, 
these  may  be  simple  markings  such  as  plurals, 
articles,  linking  words,  and  negatives  or  more 
complex  structures  such  as  tense/aspect  usage, 
case  morphology,  passive  constructions,  word 
order,  and  embedding.  (Has  been  coded  S-2  in 
some  nonautomated  applications.)  [Computer 
Code  20] 

Level  2+  (Limited  Working  Proficiency,  Plus) 

Able  to  satisfy  most  work  requirements  with 
language  usage  that  is  often,  but  not  always, 
acceptable  and  effective.  The  individual  shows 
considerable  ability  to  communicate  effectively 
on  topics  relating  to  particular  interests  and 
special  fields  of  competence.  Often  shows  a  high 
degree  of  fluency  and  ease  of  speech,  yet  when 
under  tension  or  pressure,  the  ability  to  use  the 
language  effectively  may  deteriorate. 
Comprehension  of  normal  native  speech  is 
typically  nearly  complete.  The  individual  may 
miss  cultural  and  local  references  and  may 
require  a  native  speaker  to  adjust  to  his/her 
limitations  in  some  ways.  Native  speakers  of*en 
perceive  the  individual’s  speech  to  contain 
awkward  or  inaccurate  phrasing  of  ideas, 
mistaken  time,  space,  and  person  references,  cr 
to  be  in  some  way  inappropriate,  ;f  not  strictly 
incorrect. 

Examples:  Typically  the  individual  can 
participate  in  most  social,  formal,  and  informal 
interactions,  but  limitations  either  in  range  of 
contexts,  types  of  tasks,  or  level  of  accuracy 


Kinder  effectiveness.  The  individual  may  be  ill  at 
ease  with  the  use  of  the  language  either  in  social 
interaction  or  in  speaking  at  length  in 
professional  contexts.  He/she  is  generally  strong 
in  either  structural  precision  or  vocabulary,  but 
not  in  both.  Weakness  or  unevenness  in  one  of  the 
foregoing,  or  in  pronunciation,  occasionally 
results  in  miscommunication.  Normally  controls, 
but  cannot  always  easily  produce  general 
vocabulary.  Discourse  is  often  incohesive.  (Has 
been  coded  S-2+  in  some  nonautomated 
applications.)  [Computer  Code  26] 

Level  3  (General  Professional  Proficiency) 

Able  to  speak  the  language  with  sufficient 
structural  accuracy  and  vocabulary  to  participate 
effectively  in  most  formal  and  informal 
conversations  on  practical,  social,  and 
professional  topics.  Nevertheless,  the  individual’s 
limitations  generally  restrict  the  professional 
contexts  of  language  use  to  matters  of  shared 
knowledge  and/or  international  convention. 
Discourse  is  cohesive.  The  individual  uses  the 
language  acceptably,  but  with  some  noticeable 
imperfections;  yet,  errors  virtually  never  interfere 
with  understanding  and  rarely  disturb  the  naiive 
speaker.  The  individual  can  effectively  combine 
structure  and  vocabulary  to  convey  his/her 
meaning  accurately.  The  individual  speaks 
readily  and  fills  pauses  suitably.  In  face-to-face 
conversation  with  natives  speaking  the  standard 
diaiect  at  a  normal  rate  of  speech,  comprehension 
is  quite  complete.  Although  cultural  references, 
proverbs,  and  the  implications  of  nuances  and 
idiom  may  not  be  fully  understood,  the  individual 
can  easily  repair  the  conversation.  Pronunciation 
may  be  obviously  foreign.  Individual  sounds  are 
accurate;  but  stress,  intonation,  and  pitch  control 
may  be  faulty. 

Examples:  Can  typically  discuss  particular 
interests  and  special  fields  of  competence  with 
reasonable  ease.  Can  use  the  language  as  part  of 
normal  professional  duties  such  as  answering 
objections,  clarifying  points,  justifying  decisions, 
understanding  the  essence  of  challenges,  stating 
and  defending  policy,  conducting  meetings, 
delivering  briefings,  or  other  extended  and 
elaborate  informative  monologues.  Can  reliably 
elicit  information  and  informed  opinion  from 
native  speakers.  Structural  inaccuracy  is  rarely 
the  major  cause  of  misunderstanding.  Use  of 
structural  devices  is  flexible  and  elaborate. 
Without  searching  for  words  or  phrases,  the 
individual  uses  the  language  clearly  and  relatively 
naturally  to  elaborate  concepts  freely  and  make 
ideas  easily  understandable  to  native  speakers. 
Errcs  occur  in  low-frequency  and  highly 
complex  structures.  (Has  been  coded  S-3  in  some 
nonautomated  applications.)  [Computer  Code 
30] 


Level  3+  (General  Professional  Proficiency) 

Plus) 

Is  often  able  to  use  the  language  to  satisfy 
professional  needs  In  a  wide  range  of 
sophisticated  and  demanding  tasks. 

Examples:  Despite  obvious  strengths,  may 
exhibit  some  hesitancy,  uncertainty,  effort,  or 
errors  which  limit  the  range  of  language-use  tasks 
that  can  be  reliably  performed.  Typically  there  is 
particular  strength  in  fluency  and  one  or  more, 
but  not  all,  of  the  following:  breadth  of  lexicon, 
including  low-  and  medium-frequency  items, 
especially  socio-liuguistic/cultural  references 
and  nuances  of  close  synonyms;  structural 
precision,  with  sophisticated  features  that  are 
readily,  accurately,  and  appropriately  controlled 
(such  as  complex  modification  and  embedding  in 
Indo-European  languages);  discourse 
competence  in  a  wide  range  of  contexts  and 
tasks,  often  matching  a  native  speaker’s  strategic 
and  organizational  abilities  and  expectations. 
Occasional  patterned  errors  occur  in  low 
frequency  and  highly-complex  structures.  (Has 
been  coded  S-3+  in  some  nonautomated 
applications.)  [Computer  Code  36] 

Level  4  (Advanced  Professional  Proficiency) 

Able  to  use  the  language  fluently  and 
accurately  on  all  levels  -normally  pertinent  to 
professional  needs.  The  individual's  language 
usage  and  ability  to  function  are  fully  successful. 
Organizes  discourse  well,  using  appropriate 
rhetorical  speech  devices,  native  cultural 
references,  and  understanding.  Language  ability 
only  rarely  hinders  him/her  in  performing  any 
task  requiring  language;  yet,  the  individual  would 
seldom  be  perceived  as  a  native.  Speaks 
effortlessly  and  smoothly  and  is  able  to  use  the 
language  with  a  high  degree  of  effectiveness, 
reliability,  and  precision  for  all  representational 
purposes  within  the  range  of  personal  and 
professional  experience  and  scope  of 
responsibilities.  Can  serve  as  an  informal 
interpreter  in  a  range  of  unpredictable 
circumstances.  Can  perform  extensive, 
sophisticated  language  tasks,  encompassing 
most  matters  of  interest  to  well-educated  native 
speakers,  including  tasks  which  do  not  bear 
directly  on  a  professional  specialty. 

Examples:  Can  discuss  in  detail  concepts 
which  are  fundamentally  different  from  those  of 
the  target  culture  and  make  those  concepts  clear 
and  accessible  to  the  native  speaker.  Similarly, 
the  Individual  can  understand  the  details  and 
ramifications  of  concepts  that  are  culturally  or 
conceptually  different  from  his/her  own.  Can  set 
the  tone  of  interpersonal  official,  semi-official, 
and  non-professional  verbal  exchanges  with  a 
representative  range  of  native  speakers  (in  a 
range  of  varied  audiences,  purposes,  tasks,  and 
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settings).  Can  play  an  effective  role  among  native 
speakers  in  such  contexts  as  conferences, 
lectures,  and  debates  on  matters  of  disagreement. 
Can  advocate  a  position  at  length,  both  formally 
and  in  chance  encounters,  using  sophisticated 
verbal  strategies.  Understands  and  reliably 
produces  shifts  of  both  subject  matter  and  tone. 
Can  understand  native  speakers  of  the  standard 
and  other  major  dialects  in  essentially  any  face-to- 
face  interaction.  (Has  been  coded  S-4  in  some 
nonautomated  applications.)  [Computer  Code  40] 

Level  4+  (Advanced  Professional  Proficiency, 
Pius) 

Speaking  proficiency  is  regularly  superior  in  all 
respects,  usually  equivalent  to  that  of  a  well- 
educated,  highly  articulate  native  speaker. 
Language  ability  does  not  impede  the 
performance  of  any  language-use  task.  However, 
the  individual  would  not  necessarily  be  perceived 
as  culturally  native. 

Examples;  The  individual  organizes  discourse 
well,  employing  functional  rhetorical  speech 
devices,  native  cultural  references  and 
understanding.  Effectively  applies  a  native 
speaker’s  social  and  circumstantial  knowledge. 
However,  cannot  sustain  that  performance  under 


all  circumstances.  While  the  individual  has  a  wide 
range  and  control  of  structure,  an  occasional  non¬ 
native  slip  may  occur.  The  individual  has  a 
sophisticated  control  of  vocabulary  and  phrasing 
that  is  rarely  imprecise,  yet  there  are  occasional 
weaknesses  in  idioms,  colloquialisms, 
pronunciation,  cultural  reference  or  there  may  be 
an  occasional  failure  to  interact  in  a  totally  native 
manner.  (Has  been  coded  S-4+  in  some 
nonautomated  applications.)  [Computer  Code  46] 

Level  5  (Functionally  Native  Proficiency) 

Speaking  proficiency  is  functionally  equivalent 
to  that  of  a  highly  articulate  well-educated  native 
speaker  and  reflects  the  cultural  standards  of  the 
country  where  the  language  is  natively  spoken. 

The  individual  uses  the  language  with  complete 
flexibility  and  intuition,  so  that  speech  on  all  levels 
is  fully  accepted  by  well-educated  native  speakers 
in  all  of  its  features,  including  breadth  of 
vocabulary  and  idiom,  colloquialisms,  and 
pertinent  cultural  references.  Pronunciation  is 
typically  consistent  with  that  of  well-educated 
native  speakers  of  a  non-stigmatized  dialect.  (Has 
been  coded  S-5  in  some  nonautomated 
applications.)  [Computer  Code  50] 
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